Text data processing method and device, medium and program product

Abstract

The embodiment of the invention provides a text data processing method and device, a medium and a program product. The method comprises the following steps: inputting a global feature vector corresponding to text data to be processed into a subject-object division model to obtain a subject feature vector, an object feature vector and a shared feature vector; wherein the shared feature vector is the part, except the subject feature vector and the object feature vector, in the global feature vector; inputting the subject feature vector, the object feature vector and the shared feature vector into a subject-object filtering model, performing feature fusion on the subject feature vector and the shared feature vector to obtain a filtered subject feature vector, and performing feature fusion on the object feature vector and the shared feature vector to obtain a filtered object feature vector; and obtaining a relation triple in the text data according to the filtered subject feature vector and the filtered object feature vector. According to the method, the accuracy of extracting the relation triad in the text data can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a text data processing method, device, medium and program product. Background Art

[0002] In the field of natural language processing technology, a relation triple may refer to a structural form of knowledge represented by three components, and the knowledge in this structural form may also be called a relation triple or triple.

[0003] A triple consists of three components: subject, predicate and object. The subject can also be called subject, first entity or head entity, the predicate can also be called predicate or relation, and the object can also be called object, second entity or tail entity. This structural form can be used to represent the relationship between entities. Both the subject and the object are entities, and entities can be understood as concrete things such as specific products, services or events. For example, extract the triple of the text data "A deposits money into Bank B", where "A" is the subject, "Bank B" is the object, and "deposit money" is the relation. Then "A-deposit money-Bank B" constitutes a knowledge in the form of a relational triple structure, that is, a triple is extracted from the text data.

[0004] In the existing method of processing text data to obtain relation triplets in the text data, features of the text data can be extracted through a pre-trained network model, and classification prediction is performed based on the extracted features to obtain relation triplets in the text data. However, this method has a high probability of extracting the wrong subject or object, resulting in a low accuracy of extracting relation triplets. Summary of the invention

[0005] The embodiments of the present application provide a text data processing method, device, medium and program product to improve the accuracy of extracting relation triples in text data.

[0006] In a first aspect, an embodiment of the present application provides a text data processing method, the method comprising: inputting a global feature vector corresponding to the text data to be processed into a subject-object division model to obtain a subject feature vector, an object feature vector and a shared feature vector; wherein the subject-object division model is a network model for distinguishing the subject and the object in a relation triple of the input information, the subject feature vector is the part corresponding to the subject in the global feature vector, the object feature vector is the part corresponding to the object in the global feature vector, and the shared feature vector is the part of the global feature vector other than the subject feature vector and the object feature vector; inputting the subject feature vector, the object feature vector and the shared feature vector into a subject-object filtering model, performing feature fusion on the subject feature vector and the shared feature vector to obtain a filtered subject feature vector, and performing feature fusion on the object feature vector and the shared feature vector to obtain a filtered object feature vector; wherein the subject-object filtering model is a network model for performing feature fusion on the input information; and obtaining the relation triple in the text data according to the filtered subject feature vector and the filtered object feature vector.

[0007] In a possible implementation, the subject-object division model includes a subject gate and an object gate, the subject gate is a sub-model in the subject-object division model for determining a subject feature vector through a gating mechanism, and the object gate is a sub-model in the subject-object division model for determining an object feature vector through a gating mechanism; the step of inputting a global feature vector corresponding to the text data to be processed into the subject-object division model to obtain a subject feature vector, an object feature vector and a shared feature vector comprises: inputting the global feature vector corresponding to the text data to be processed into the subject gate to obtain a subject feature vector, and inputting the global feature vector corresponding to the text data to be processed into the object gate to obtain an object feature vector; and determining the portion of the global feature vector other than the subject feature vector and the object feature vector as the shared feature vector.

[0008] In a possible implementation manner, the subject gate satisfies the formula when processing the global feature vector: Among them, x t represents the tth element in the global feature vector, h t-1 represents the hidden state at the previous moment, represents the weight matrix of the subject gate, represents the weight matrix of the hidden state of the subject gate, represents the bias vector of the main gate, cummax represents the cumulative sum operation, represents the feature vector corresponding to the tth element in the subject feature vector; the object gate satisfies the formula when processing the global feature vector: in, represents the weight matrix of the object gate, The weight matrix representing the hidden state of the object gate, represents the bias vector of the object gate, Represents the feature vector corresponding to the tth element in the object feature vector.

[0009] In a possible implementation, the feature fusion of the subject feature vector and the shared feature vector to obtain the filtered subject feature vector includes: adding the subject feature vector and the shared feature vector and then performing data processing using a hyperbolic tangent activation function to obtain the filtered subject feature vector; the feature fusion of the object feature vector and the shared feature vector to obtain the filtered object feature vector includes: adding the object feature vector and the shared feature vector and then performing data processing using a hyperbolic tangent activation function to obtain the filtered subject feature vector.

[0010] In a possible implementation, the obtaining of the relation triples in the text data based on the filtered subject feature vector and the filtered object feature vector includes: processing the shared feature vector through data of a hyperbolic tangent activation function to obtain a processed shared feature vector; inputting a feature vector obtained by connecting the filtered subject feature vector and the processed shared feature vector into a subject extraction model to obtain the subject in the text data, wherein the subject extraction model is a network model for decoding input information; inputting a feature vector obtained by connecting the filtered object feature vector and the processed shared feature vector into an object extraction model to obtain the object in the text data, wherein the object extraction model is a network model for decoding input information; and obtaining the relation triples in the text data based on the subject and the object.

[0011] In a possible implementation, the method further includes: inputting the text data to be processed into an encoder for performing feature vector encoding on input information to obtain a text feature vector corresponding to the text data; fusing the text feature vector with multiple pre-encoded relationship feature vectors using a multi-head attention mechanism to obtain a global feature vector corresponding to the text data, wherein any of the relationship feature vectors is used to represent a relationship in a relationship triplet.

[0012] In a possible implementation, the relationship triples are used to perform public opinion analysis on a bank, and the method further includes: obtaining text data associated with the bank; performing data preprocessing on the text data associated with the bank to obtain preprocessed text data, wherein the data preprocessing includes at least one of character correction and text length adjustment; and determining the preprocessed text data as text data to be processed.

[0013] In a second aspect, an embodiment of the present application provides a text data processing device, the device comprising: a first processing module, used to input a global feature vector corresponding to the text data to be processed into a subject-object division model to obtain a subject feature vector, an object feature vector and a shared feature vector; wherein the subject-object division model is a network model used to distinguish the subject and the object in a relation triplet of the input information, the subject feature vector is the part of the global feature vector corresponding to the subject, the object feature vector is the part of the global feature vector corresponding to the object, and the shared feature vector is the part of the global feature vector excluding the subject feature vector and the object feature vector. The part outside the feature vector; the first processing module is also used to input the subject feature vector, the object feature vector and the shared feature vector into the subject-object filtering model, perform feature fusion on the subject feature vector and the shared feature vector to obtain the filtered subject feature vector, and perform feature fusion on the object feature vector and the shared feature vector to obtain the filtered object feature vector; wherein the subject-object filtering model is a network model for performing feature fusion on input information; the second processing module is used to obtain the relationship triples in the text data according to the filtered subject feature vector and the filtered object feature vector.

[0014] In a possible implementation, the subject-object division model includes a subject gate and an object gate, the subject gate is a sub-model in the subject-object division model used to determine the subject feature vector through a gating mechanism, and the object gate is a sub-model in the subject-object division model used to determine the object feature vector through a gating mechanism; the first processing module is specifically used to: input the global feature vector corresponding to the text data to be processed into the subject gate to obtain the subject feature vector, and input the global feature vector corresponding to the text data to be processed into the object gate to obtain the object feature vector; determine the part of the global feature vector other than the subject feature vector and the object feature vector as the shared feature vector.

[0015] In a possible implementation manner, the subject gate satisfies the formula when processing the global feature vector: Among them, x t represents the tth element in the global feature vector, h t-1 represents the hidden state at the previous moment, represents the weight matrix of the subject gate, represents the weight matrix of the hidden state of the subject gate, represents the bias vector of the main gate, cummax represents the cumulative sum operation, represents the feature vector corresponding to the tth element in the subject feature vector; the object gate satisfies the formula when processing the global feature vector: in, represents the weight matrix of the object gate, The weight matrix representing the hidden state of the object gate, represents the bias vector of the object gate, Represents the feature vector corresponding to the tth element in the object feature vector.

[0016] In a possible implementation, the first processing module is specifically used to: add the subject feature vector and the shared feature vector, and then perform data processing using a hyperbolic tangent activation function to obtain the filtered subject feature vector; the first processing module is specifically used to: add the object feature vector and the shared feature vector, and then perform data processing using a hyperbolic tangent activation function to obtain the filtered subject feature vector.

[0017] In a possible implementation, the second processing module is specifically used to: obtain a processed shared feature vector by processing the shared feature vector through a hyperbolic tangent activation function; input the feature vector obtained by connecting the filtered subject feature vector and the processed shared feature vector into a subject extraction model to obtain the subject in the text data, wherein the subject extraction model is a network model for decoding input information; input the feature vector obtained by connecting the filtered object feature vector and the processed shared feature vector into an object extraction model to obtain the object in the text data, wherein the object extraction model is a network model for decoding input information; and obtain a relation triple in the text data based on the subject and the object.

[0018] In a possible implementation, the first processing module is further used to: input the text data to be processed into an encoder for feature vector encoding of input information to obtain a text feature vector corresponding to the text data; fuse the text feature vector with multiple pre-encoded relationship feature vectors through a multi-head attention mechanism to obtain a global feature vector corresponding to the text data, and any of the relationship feature vectors is used to represent a relationship in a relationship triplet.

[0019] In a possible implementation, the relationship triples are used to perform public opinion analysis on a bank, and the first processing module is also used to: obtain text data associated with the bank; perform data preprocessing on the text data associated with the bank to obtain preprocessed text data, wherein the data preprocessing includes at least one of character correction and text length adjustment; and determine the preprocessed text data as the text data to be processed.

[0020] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the processor executes the first aspect above and / or various possible implementations of the first aspect.

[0021] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementations of the first aspect.

[0022] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0023] The text data processing method, device, medium and program product provided by the embodiments of the present application are as follows. The method performs feature division by inputting the global feature vector corresponding to the text data to be processed into the subject-object division model to obtain the subject feature vector, the object feature vector and the shared feature vector, wherein the shared feature vector is the part of the global feature vector other than the subject feature vector and the object feature vector. Since the shared feature vector may contain information representing the subject, information representing the object or information representing the subject-object relationship, the shared feature vector is a feature vector that helps to enhance the information representation, and this part of the feature vector is not discarded in the subsequent processing; further, by inputting the subject feature vector, the object feature vector and the shared feature vector into the subject-object filtering model, the subject feature vector and the shared feature vector can be feature fused, so that the expression of the subject information in the subject feature vector is enhanced by the shared feature vector and the expression of the object information is filtered out, thereby improving the representation strength of the subject information by the subject feature vector after filtering; the object feature vector and the shared feature vector can be feature fused, so that the expression of the object information in the object feature vector is enhanced by the shared feature vector and the expression of the subject information is filtered out, thereby improving the representation strength of the object feature vector by the filtered object feature vector. Based on the filtered subject feature vector and the filtered object feature vector with enhanced representation strength, the probability of subject determination error, object determination error or relationship matching error when obtaining relationship triples can be reduced, thereby improving the accuracy of extracting relationship triples. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0025] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0026] Figure 2 Schematic diagram of the text data processing method provided in the embodiment of the present application Figure 1 ;

[0027] Figure 3 A schematic diagram of a subject-object encoding module provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of the division and filtering process provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of the structure of a text-relation encoding module provided in an embodiment of the present application;

[0030] Figure 6 Schematic diagram of the text data processing method provided in the embodiment of the present application Figure 2;

[0031] Figure 7 A schematic diagram of the structure of a text data processing device provided in an embodiment of the present application;

[0032] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0033] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0034] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0035] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same functions and effects. For example, the first electronic device and the second electronic device are only used to distinguish different electronic devices, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0036] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0037] In the embodiments of the present application, "at least one" means one or more, and "more" means two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0038] In the technical solutions of the embodiments of the present application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0039] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0040] With the development of the Internet and social media, the public can express their opinions and emotions about the products or services of an organization through the Internet. The text data carrying these opinions and emotions is an effective reference for organizations to analyze whether their products or services meet user needs.

[0041] Taking banks as an example, users' opinions and emotional expressions on banks, financial products or services are massive, multi-source and dynamic. For banks, if they can accurately extract useful information from these massive text data, it will be of great significance for them to understand market dynamics, optimize product services and manage public relations crises.

[0042] However, existing text data processing methods, such as keyword search methods and sentiment analysis methods, have limitations in deeply understanding text content and capturing specific opinion expressions. For example, keyword search methods rely on predefined keyword lists, which cannot capture complex relationships in text data. Sentiment analysis methods are usually based on dictionaries or machine learning models. Although they can identify the overall sentiment tendency of text, they are difficult to capture specific opinion expressions and relationships between entities.

[0043] For example, the method based on sentiment analysis usually takes the comment text as a unit and uses dictionary matching to analyze the sentiment tendency of the content of the comment text. For example, three sentiment types are set: positive sentiment, neutral sentiment, and negative sentiment. Based on the positive and negative sentiment dictionary, the positive words in the comment text are given an initial weight of 1, and the negative words are given an initial weight of -1, and then the sum of the sentiment weights of the comment text is calculated, and the sentiment analysis result corresponding to the comment text is obtained.

[0044] When this method performs sentiment analysis, it ignores the complexity and multi-dimensionality of sentiment, resulting in oversimplification of sentiment analysis results. For example, the sentiment analysis results can only indicate that the overall sentiment tendency of the comment text is positive, negative or neutral, and it is difficult to clearly determine the specific object or subject to which the sentiment points. This makes it difficult to capture the specific sentiment tendency of users towards the products, services or events of the organization in the comment text. Therefore, the sentiment analysis results are less sophisticated. In fact, the sentiment analysis results cannot indicate the truth in the comment text. For example, a comment text may contain both praise for one aspect and criticism for another. In this case, the sentiment analysis result may be neutral sentiment, which is inconsistent with the true sentiment in the comment text.

[0045] Furthermore, this method cannot pay attention to the entities in the comment text and the relationships between entities. Therefore, it cannot distinguish the sentiment differences of the same entity in different aspects, such as the product design of a product and the customer service of the product. The relationship information between entities has a significant impact on the accuracy and complexity of understanding public opinion. For long texts containing multiple sentences or paragraphs, if the entities and the relationships between entities in the comment text cannot be accurately determined, sentiment analysis will find it difficult to accurately capture the overall sentiment tendency in the comment text. When the sentiment changes in the text, it is even more difficult to obtain a fine-grained and accurate analysis result. Therefore, this method is difficult to handle long texts and text data with complex structures.

[0046] In order to better process text data and realize fine-grained analysis of text data, natural language text data can be processed through the method of relation triple extraction to obtain one or more relation triples, and then by analyzing the subject, object and the relationship between the two in the relation triples, the public opinion information such as the opinions, emotions and attitudes expressed by users towards the entities related to the organization can be understood, thereby improving the accuracy and granularity of the organization's acquisition of public opinion information.

[0047] The method for extracting relational triples can process natural language text data, identify and extract "subject-relationship-object" relational triples in text data. In some scenarios of the embodiments of the present application, the text data may be text content obtained from the Internet or on-site, or text content obtained after conversion from voice content, etc. The subject in the relational triple may be, for example, a user such as a user, the object may be, for example, a service provider such as an institution, and the relationship may be, for example, the subject's actions, emotions, claims and / or intentions to the object. For example, a text data is an evaluation text of "a user expresses his likes and dislikes for a certain institution", then from the evaluation text, a "user-emotion-institution" type relational triple may be extracted, based on the relational triple, the user's emotions towards the institution can be analyzed, for example, the user's emotions towards the institution can be analyzed, such as whether it is indifferent, like, dislike, angry, disappointed or other emotions, based on which, the institution can understand the user's likes and dislikes of itself in a more fine-grained manner.

[0048] Taking the text data associated with banks as an example, by extracting the relational triples of the text data "Mr. A is willing to deposit money in Bank B, but the staff of Bank B refused to process it for Mr. A.", we can obtain two relational triples: "Mr. A - willing to deposit money - Bank B" and "The staff of Bank B - refused to process it - to Mr. A". If the text data is processed based on the above sentiment analysis method, the overall sentiment analysis result may be neutral due to the offsetting effect of the two key words "willing" and "refused" with different sentiment tendencies. However, after obtaining the above two relational triples through the method of relational triple extraction and analyzing each relational triple, we can obtain the complex relationship between the entities, and then obtain public opinion information that is more in line with the user's real emotions.

[0049] By extracting relational triples, we can more accurately understand the relationships between entities in text data, thereby providing a richer and more sophisticated information basis for advanced text data analysis tasks such as sentiment analysis and topic identification. The relational triple extraction method can not only identify entities in text data and the relationships between them, but also capture the complex relationships and subtle sentiment changes in text data, providing a more in-depth public opinion analysis. In addition, the relational triple extraction method can be combined with natural language processing and machine learning methods to maintain efficiency and accuracy when processing large amounts of unstructured text data. For institutions such as banks, it can enable banks to timely obtain and analyze massive amounts of text data related to bank public opinion.

[0050] In the existing methods for extracting relational triples from text data, a neural network model that can embed text data is usually used to extract global features from text data to obtain a global feature vector that represents the text data, and then the global feature vector is input into a pre-trained keyword prediction model to predict the part representing the keyword, and then the predicted keyword part is decoded to extract the keywords corresponding to the entity and the relationship keywords, and then the keywords are arranged according to the position of each keyword in the text data to obtain one or more relationship triples. When the model in this method processes the global feature vector, it does not pay too much attention to the connection between the input elements of each part, so it does not fully mine the association information between the elements of each part, so that the network model cannot deeply understand and analyze the semantic association of each part in the text data, resulting in the recognition error of the keywords corresponding to the subject, object or relationship or the matching error between the entity relationship, which increases the error probability of the obtained relationship triples and reduces the accuracy of extracting the relationship triples.

[0051] In view of this, an embodiment of the present application provides a text data processing method, which inputs the global feature vector corresponding to the text data into a subject-object division model for feature division and obtains three parts, namely, a subject feature vector, an object feature vector and a shared feature vector, wherein the shared feature vector is the part of the global feature vector other than the subject feature vector and the object feature vector. Since the shared feature vector may contain information representing the subject, information representing the object, or information representing the subject-object relationship, the shared feature vector is a feature vector that helps to enhance the information representation, and this part of the feature vector is not discarded in subsequent processing; further, by inputting the subject feature vector, the object feature vector and the shared feature vector into the subject-object filtering model, the subject feature vector and the shared feature vector can be feature fused, so that the expression of the subject information in the subject feature vector is enhanced through the shared feature vector and the expression of the object information is filtered out, thereby improving the representation strength of the subject feature vector for the subject information after filtering; the object feature vector and the shared feature vector can be feature fused, so that the expression of the object information in the object feature vector is enhanced through the shared feature vector and the expression of the subject information is filtered out, thereby improving the representation strength of the object feature vector for the object information after filtering. Based on the filtered subject feature vector and the filtered object feature vector with enhanced representation strength, the probability of subject determination error, object determination error or relationship matching error when obtaining relationship triples can be reduced, thereby improving the accuracy of extracting relationship triples. Based on this, highly granular and accurate text data analysis can be achieved to obtain more detailed and reliable public opinion information.

[0052] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application, such as Figure 1As shown, a large amount of text data associated with Bank A can be obtained from Internet nodes such as social networking sites or posting platforms, such as text data of comments related to Bank A's products, Bank A's outlets, Bank A's services, and Bank A-related events.

[0053] When processing these text data, any of these text data is input into a feature extraction model for tokenization, and paragraphs or sentences are decomposed into words or subwords to obtain multiple tokens, wherein the feature extraction model can be a neural network model for encoding natural language text data. Through the feature extraction model, each token can be embedded (Embedding) to obtain the feature vector corresponding to the text data, and the conversion from text data to vector (numerical array) is realized, so that the triple extraction model can understand, perceive and process the text data, wherein the triple extraction model can be a pre-trained neural network model for extracting relational triples.

[0054] After obtaining the feature vector corresponding to the text data, it can be input into the triple extraction model to process the feature vector to obtain the relationship triples contained in the text data. In some implementations, the triple extraction model and the feature extraction model can also be two parts integrated in a neural network model. After obtaining each triple, each relationship triple can be understood and analyzed by the method of public opinion analysis to obtain the public opinion information about Bank A contained in the text data. For example, analyze the public opinion information such as the user's attitude, opinions, usage experience and improvement suggestions for Bank A in the text data, so that Bank A can improve its products or services through these public opinion information.

[0055] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0056] Figure 2 Schematic diagram of the text data processing method provided in the embodiment of the present application Figure 1 The execution subject of the method may be an electronic device with corresponding data storage and computing capabilities, such as a computer, a server or a server cluster. Figure 2 As shown, the method includes:

[0057] S201, input the global feature vector corresponding to the text data to be processed into the subject-object division model to obtain the subject feature vector, the object feature vector and the shared feature vector; wherein the subject-object division model is a network model used to distinguish the subject and the object in the relation triple of the input information, the subject feature vector is the part of the global feature vector corresponding to the subject, the object feature vector is the part of the global feature vector corresponding to the object, and the shared feature vector is the part of the global feature vector excluding the subject feature vector and the object feature vector.

[0058] Exemplarily, the text data may be any natural language text data, such as a comment text or event description text obtained from the Internet. The global feature vector may be a feature representation of the text data at the level of the entire text data obtained after word segmentation and embedding of the text data, and the global feature vector may semantically express the text data in the feature space in the form of a vector. The obtained global features help the neural network model of machine learning to understand the overall content and structure of the text data, rather than being limited to understanding only the local word relationship in the text data.

[0059] The subject-object division model may be a network model for distinguishing the subject and the object in the relation triple of the input information. For example, the model may be obtained by training the initial network model in a supervised training manner.

[0060] Exemplarily, the subjects and objects in multiple training text data are marked. For example, for the words corresponding to the entity, the start token and the end token corresponding to the entity can be marked as 1, and the other tokens between the two can be marked as 0; for the words corresponding to the non-entity, the corresponding tokens can be marked as 0, and the training text data and its corresponding labels are obtained. The training text data is input into the initial network model for prediction processing, and the current prediction result is output. The loss function value corresponding to the current training round is calculated based on the predicted subject and object and the label in the prediction result, and the back propagation calculation is performed based on the loss function value to adjust the model parameters to achieve a round of model training. After multiple rounds of training, if the loss function value of the current round meets the convergence condition, the model corresponding to the model parameters of the current round can be a trained subject-object segmentation model.

[0061] Based on the subject-object division model, the global feature vector corresponding to the text data to be processed can be processed to obtain a subject feature vector, an object feature vector and a shared feature vector. It should be noted that if the trained subject-object division model can obtain the subject and the object through decoding, the subject feature vector can be the hidden state vector of the subject corresponding to the hidden layer before the subject is decoded, and the object feature vector can be the hidden state vector of the object corresponding to the hidden layer before the object is decoded.

[0062] Exemplarily, in a neural network model, sequence data information such as natural language and / or time series can be processed based on a gating mechanism. The gating mechanism can control the flow of information and the update of the state. For example, a gating mechanism is used in a long short term memory (LSTM) network, and different types of gates (such as input gates, forget gates, and output gates) are introduced to determine which information should be retained, updated, or discarded through each gate, thereby more effectively capturing long-range dependencies in sequence data. After capturing these long-range dependencies, the representation of feature vectors can be effectively enhanced, thereby improving the accuracy of the output information.

[0063] In a possible implementation, the subject-object division model includes a subject gate and an object gate, wherein the subject gate is a sub-model in the subject-object division model for determining a subject feature vector through a gating mechanism, and the object gate is a sub-model in the subject-object division model for determining an object feature vector through a gating mechanism. When the global feature vector corresponding to the text data to be processed is input into the subject-object division model to obtain a subject feature vector, an object feature vector, and a shared feature vector, the specific method may be: inputting the global feature vector corresponding to the text data to be processed into the subject gate to obtain a subject feature vector, and inputting the global feature vector corresponding to the text data to be processed into the object gate to obtain an object feature vector; and determining the portion of the global feature vector other than the subject feature vector and the object feature vector as a shared feature vector.

[0064] Exemplarily, the subject-object division model can also be called a subject-object coding module. The subject gate and object gate included in the subject-object coding module can be two sub-models with the same model structure or two sub-models with different model structures, and the two can have their own corresponding model parameters. The subject gate can extract the subject feature vector from the input global feature vector through a preset activation function, and the object gate can also use the same activation function to extract the object feature vector from the input global feature vector. After obtaining the subject feature vector and the object feature vector, the part of the global feature vector other than the subject feature vector and the object feature vector can be determined as a shared feature vector.

[0065] In one possible implementation, the main gate satisfies the formula when processing the global feature vector: Among them, x t represents the tth element in the global eigenvector, h t-1 represents the hidden state at the previous moment, represents the weight matrix of the main gate, The weight matrix representing the hidden state of the main gate, represents the bias vector of the main gate, cummax represents the cumulative sum operation, Represents the feature vector corresponding to the tth element in the subject feature vector; the object gate satisfies the formula when processing the global feature vector: in, represents the weight matrix of the object gate, The weight matrix representing the hidden state of the object gate, represents the bias vector of the object gate, Represents the feature vector corresponding to the tth element in the object feature vector.

[0066] Exemplarily, cummax can be an activation function representing a cumulative sum operation. For example, gate = cummax(·) = cumsum(softmax(·)), and accordingly, cummax(x1,x2,...,x n )=(x1,x1+x2,...,x1+x2+...+x n ), where gate can be understood as a subject gate or an object gate, and x n It can be understood as the information corresponding to the nth token in the global feature vector.

[0067] After the activation function is operated, a vector can be obtained with 0 and 1 representing the value of the vector element. Assuming the preset threshold is 0.5, based on cummax(x1,x2,...,x n )=(x1,x1+x2,...,x1+x2+...+x n ) After calculation, if the calculated value of the nth element is greater than or equal to the threshold value 0.5, the vector element value of the element is represented as 1 in the gate representation vector. If the calculated value of the nth element is less than the threshold value 0.5, the vector element value of the element is represented as 0 in the gate representation vector, and the gate representation vector can be obtained. That is, the output form of the cummax activation function can be expressed as gate = (0,...,0,1,...,1), where the ellipsis between 0 and 0 represents 0, and the ellipsis between 1 and 1 represents 1. If 1 represents the vector element corresponding to the entity, the part of the global feature vector corresponding to the continuous 1 is the feature vector corresponding to the object or the feature vector corresponding to the subject.

[0068] Based on this, the state in the global feature vector can be divided into a part represented by 0 and a part represented by 1 by cummax. After that, different update rules can be applied to update the subject feature vector and the object feature vector in the subject-object segmentation model. Different update rules can be understood as update rules set for the subject gate and the object gate respectively. For example, the update rule for the subject gate can be set as gate = (0, ..., 0, 1, ..., 1), and the update rule for the object gate can be set as gate = (1, ..., 1, 0, ..., 0).

[0069] For example, the main door and object gate The information in the global feature vector is divided into two segments, and then the subject feature vector, object feature vector and shared feature vector are obtained. and middle, It can also be understood as the subject feature vector corresponding to the tth token obtained through the subject gate; It can also be understood as the object feature vector corresponding to the tth token obtained through the object gate; x t It can also be understood as the information corresponding to the tth token in the global feature vector; h t-1 It can also be understood as the hidden layer state obtained based on the tokens before the tth token, that is, the hidden state at the previous moment. The hidden state can also be understood as the hidden state vector. and can be model parameters obtained through training. In the processing process, the time corresponding to processing each token can be understood as a time step. For example, the time corresponding to processing the information corresponding to the nth token is the nth time step, which can be expressed as t n .

[0070] Exemplarily, different update rules can be adopted through the subject gate or the object gate to update the feature information of a specific task of determining the subject feature vector or determining the object feature vector, and finally obtain three specific partition representations of subject partition, object partition and partition, and the three specific partitions are respectively the current subject feature vector, the current object feature vector and the current shared feature vector. As shown in the following formula, the subject partition ρ can be obtained s , object partition ρ o , shared partition ρ so .

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] in, Represents the shared partition corresponding to the previous time step; Indicates the main partition corresponding to the previous time step; represents the object partition corresponding to the previous time step; Represents the subject feature vector output by the subject gate corresponding to the previous time step; represents the object feature vector output by the object gate corresponding to the previous time step; c t-1 Represents the current feature vector corresponding to the previous time step; Represents the candidate values ​​corresponding to the subject gate and the object gate at the current time step, which can be obtained by the hyperbolic tangent function; Indicates the information currently determining the subject feature vector input; Indicates the information of the current determined object characteristic vector input; Represents the information currently determining the shared feature vector input.

[0078] Based on this, the subject and object in the global feature vector can be determined by adopting the subject gate and object gate of the gating mechanism, and in the determination process, the long-term dependent information in the global feature vector can be captured through the gating mechanism, so that the information between the subject, object and sharing can be interacted, so that useful information can be retained, useless information can be discarded, and the accuracy of the determined subject feature vector and object feature vector can be improved.

[0079] S202, inputting the subject feature vector, the object feature vector and the shared feature vector into the subject-object filtering model, performing feature fusion on the subject feature vector and the shared feature vector to obtain the filtered subject feature vector, and performing feature fusion on the object feature vector and the shared feature vector to obtain the filtered object feature vector; wherein the subject-object filtering model is a network model used to perform feature fusion on input information.

[0080] Exemplarily, the subject-object filtering model may be a network model for performing feature fusion on the subject feature vector, the object feature vector and the shared feature vector. The subject-object filtering model may be obtained by training. Feature fusion may be to perform, for example, convolution operation, addition operation, connection operation or other operation on the feature vector to be processed.

[0081] There may be some information about the relationship between the subject and the object in the shared feature vector. Therefore, by performing feature fusion on the subject feature vector and the shared feature vector to obtain the filtered subject feature vector, the information related to the relationship in the subject feature vector can be strengthened and the information unrelated to the relationship in the subject feature vector can be weakened. By performing feature fusion on the object feature vector and the shared feature vector to obtain the filtered object feature vector, the information related to the relationship in the object feature vector can be strengthened and the information unrelated to the relationship in the object feature vector can be weakened.

[0082] In a possible implementation, when the subject feature vector and the shared feature vector are subjected to feature fusion to obtain the filtered subject feature vector, it can be specifically implemented in the following way: the subject feature vector and the shared feature vector are added and then the data is processed by the hyperbolic tangent activation function to obtain the filtered subject feature vector; when the object feature vector and the shared feature vector are subjected to feature fusion to obtain the filtered object feature vector, it can be specifically implemented in the following way: the object feature vector and the shared feature vector are added and then the data is processed by the hyperbolic tangent activation function to obtain the filtered subject feature vector.

[0083] Exemplarily, the subject-object division model and the subject-object filtering model may be included in the subject-object encoding module.

[0084] Figure 3 A schematic diagram of a subject-object encoding module provided in an embodiment of the present application, such as Figure 3 As shown, t1, t2, t3...t n To represent different time steps, each time step will process the information corresponding to a token in the global feature vector. For example, t n The time step processes the information corresponding to the tth token in the global feature vector. The semicircle represents the object feature vector, the square represents the subject feature vector, and the parallelogram represents the shared feature vector. At each time step, the subject-object encoding module divides and filters the information corresponding to the corresponding token in the global feature vector to update and obtain the updated subject feature vector, object feature vector, and shared feature vector at the current time step.

[0085] In the filtering step of the subject-object filtering model, each feature vector obtained after the current filtering can be stored in a memory block for storing the corresponding feature vector.

[0086] Figure 4 The following is a schematic diagram of the process of division and filtering provided in the embodiment of the present application. Figure 4 As shown, the memory block includes the main memory block μ s , object memory block μ o and the shared memory block μ soIn μ s In the above example, the information of the specific subject partition and the shared partition is fused to obtain the subject-related partition μ s , thereby filtering out the information in the object partition. o The same logic applies. The information about the subject can also be filtered out, and the remaining feature information is retained to obtain the object-related partition μ o For μ so , the shared partition representation obtained in the partitioning step can be directly used. The information in the shared memory block can be accessed by the subject extraction task and the object extraction task, which can be expressed by the formula: μ s =ρ s +ρ so ;μ o =ρ o +ρ so ;μ so =ρ so Based on this, it is possible to not discard any useful information and improve the representation strength of the filtered feature vector.

[0087] After adding the subject feature vector and the shared feature vector, the filtered subject feature vector can be obtained through data processing of the hyperbolic tangent activation function; similarly, after adding the object feature vector and the shared feature vector, the filtered subject feature vector can be obtained through data processing of the hyperbolic tangent activation function.

[0088] For example, after each information stored in the memory block is updated, a feature vector h related to the subject can be generated. s , object-related feature vector h o and the shared eigenvector h so , the data processing of the hyperbolic tangent activation function can be expressed as the following formula: s =tanh(μ s );h o =tanh(μ o );h so =tanh(μ so ). Among them, h s represents the main feature vector after filtering, h o represents the object feature vector after filtering, h so Represents the shared feature vector after processing. The filtering process can also be understood as a denoising process.

[0089] After partitioning and denoising, c can be obtained through the information stored in the three memory blocks. t , that is, the current feature vector, which can be understood as the feature vector obtained at the current time step, consisting of the subject feature vector, the shared feature vector, and the object feature vector.t The hidden layer state h of the current time step can be generated t , which is the current hidden state vector, can be used to divide and filter the tokens of the next time step. It can be expressed by the formula: t =Linear([μ s，t ;μ o，t ;μ so，t ]), where Linear represents the linear layer, μ s，t represents the subject-related partition corresponding to time step t, μ o，t represents the object-related partition corresponding to time step t, μ so，t Represents the shared partition corresponding to the t time step. For the current feature vector c t The data processing of the hyperbolic tangent activation function can obtain the hidden layer state h t , that is, h t =tanh(c t ).

[0090] Based on this, feature fusion is achieved by adding the subject feature vector and the shared feature vector, the object feature vector and the shared feature vector respectively, and then processing the data through the hyperbolic tangent activation function. The nonlinear activation of the hyperbolic tangent activation function tanh can improve the representation of the fused feature vector. In the subject-object encoding module, the two steps of division and filtering can enhance the semantic mining of the model for each information in the text data, and at the same time enhance the interaction between the semantic vectors of the subject and the object in the triple extraction, and improve the accuracy of the model in distinguishing the subject and the object.

[0091] S203, obtaining the relation triples in the text data according to the filtered subject feature vector and the filtered object feature vector.

[0092] Exemplarily, after obtaining the filtered subject feature vector and the filtered object feature vector, a model capable of decoding and classifying the feature vectors can be used for processing, and the subject corresponding to the filtered subject feature vector and the object corresponding to the filtered object feature vector can be obtained by classification prediction.

[0093] For example, a pre-trained classifier model can be used to process the filtered subject feature vector and decode to obtain one or more subjects in the text data to be processed. The pre-trained classifier model can also be used to process the filtered object feature vector and decode to obtain one or more objects in the text data to be processed.

[0094] For another example, a subject classifier model that can classify and predict subjects and an object classifier model that can classify and predict objects can be pre-trained, and the filtered subject feature vectors can be input into the subject classifier model respectively, and one or more subjects in the text data to be processed can be classified and predicted. The subject can be a subject corresponding to a preset relationship. If it is assumed that m relationships are preset, the subject corresponding to each relationship can be predicted. For example, m subjects can be predicted, and the m subjects correspond to the m relationships respectively; the filtered object feature vectors can be input into the object classifier model, and one or more objects in the text data to be processed can be classified and predicted. The object can correspond to the predicted subject and the relationship corresponding to the subject. For example, m objects can be predicted, corresponding to m subjects and m relationships respectively.

[0095] In a possible implementation, when obtaining relation triplets in text data based on filtered subject feature vectors and filtered object feature vectors, it can be specifically implemented in the following manner: processing the shared feature vector through data processing of a hyperbolic tangent activation function to obtain a processed shared feature vector; inputting the feature vector obtained by connecting the filtered subject feature vector and the processed shared feature vector into a subject extraction model to obtain the subject in the text data, where the subject extraction model is a network model for decoding input information; inputting the feature vector obtained by connecting the filtered object feature vector and the processed shared feature vector into an object extraction model to obtain the object in the text data, where the object extraction model is a network model for decoding input information; based on the subject and the object, obtaining relation triplets in the text data.

[0096] Exemplarily, the subject extraction model and the object extraction model may be models included in a triple extraction module. The triple extraction module may be a neural network model that performs triple extraction on input information.

[0097] In the training and reasoning phases of the triple extraction module, the h obtained at each time step can be s Accumulate and get h sub , can be understood as the filtered main feature vector obtained by processing all tokens in the global feature vector. Similarly, h obtained at each time step can be o Accumulate and get h obj , which can be understood as the filtered object feature vector obtained by processing all tokens in the global feature vector. Similarly, h obtained at each time step is so After accumulation, the processed shared feature vector h can be obtained share . It can be expressed by the formula: Among them, concat can be a connection operation that accumulates connections. represents the h corresponding to the nth time step s , represents the h corresponding to the nth time step o , represents the h corresponding to the nth time step so .

[0098] During the training phase, h sub With h share The final feature vector input to the subject extraction model is obtained by combining. The subject extraction model can output the predicted subject vector based on the input. The subject vector is the filtered subject feature vector of the text data sample obtained in the training phase. The feature vector input to the subject extraction model can be expressed as follows: sub_share =concat([h sub ;h share ]), concat can be a concatenation operation that adds elements at the same position, h sub_share Indicates h sub and h share The final feature vector obtained after combination is input into the subject extraction model.

[0099] You can use h obj With h share The feature vector finally input to the object extraction model is obtained by combining. The object extraction model can output the predicted object vector based on the input. The object vector is the filtered object feature vector of the text data sample obtained in the training phase. The feature vector input to the object extraction model can be expressed by the following formula: obj_share =concat([h obj ;h share ]), concat can be a concatenation operation that adds elements at the same position, h obj_share Indicates h obj and h share The final feature vector obtained after combination is input into the object extraction model.

[0100] The predicted subject vector is compared with the real subject label to calculate the loss value of the subject; the predicted object vector is compared with the real object label to calculate the loss value of the object. According to the loss value of the subject, the model parameters of the subject-object encoding module and / or the global feature vector extraction model can be adjusted through back propagation to optimize the information extraction of the subject. According to the loss value of the object, the model parameters of the subject-object encoding module and / or the global feature vector extraction model can also be adjusted through back propagation to optimize the information extraction of the object.

[0101] For example, in the reasoning stage of the triple extraction module, h subDecode and extract h through the object extraction model obj Decoding is performed to achieve the purpose of converting the digital pointer into a relational triplet, and the relational triplet in the text data to be processed is obtained.

[0102] For example, the subject extraction model is used to extract h sub Decoding is performed, and m subjects of initial test are predicted, where m can be any positive integer, for example, m is 20. Each subject of the initial test corresponds to a first probability value that represents the credibility. For example, the larger the first probability value, the greater the probability that the corresponding subject is the correct subject, and vice versa. Moreover, each of the 20 subjects of the initial test corresponds to a relationship, which can be one of the preset relationship sets. The first probability values ​​of the 20 subjects of the initial test are compared by using a preset probability threshold, and the subjects of the initial test corresponding to the first probability value less than the probability threshold are removed, and the subjects of the initial test corresponding to the first probability value greater than or equal to the probability threshold are retained. For example, 5 subjects of the initial test are retained from the 20 subjects of the initial test, and then 5 subjects of the initial test and the corresponding relationships of the 5 subjects of the initial test can be obtained. The object extraction model is used to extract h obj Decoding is performed to predict the objects corresponding to the five initially tested subjects and the second probability value corresponding to each object. The second probability value is used to characterize the credibility of the corresponding object. Each second probability value is compared by a probability threshold that is the same as or different from the probability threshold for judging the first probability value, and the objects corresponding to the second probability values ​​that are less than the probability threshold are removed, and the objects corresponding to the second probability values ​​that are greater than or equal to the probability threshold are retained. In this way, when there is a second probability value greater than or equal to the probability threshold, at least one relationship triplet can be obtained, and each relationship triplet obtained can include a subject, a relationship corresponding to the subject, and an object corresponding to the subject. Based on this, the feature vector h representing the semantics of the subject and object obtained by the above-mentioned subject extraction model and object extraction model is extracted through the triple extraction module. sub and h obj By decoding, we can obtain the relation triples used for public opinion analysis.

[0103] The text data processing method provided by the embodiment of the present application is characterized by inputting the global feature vector corresponding to the text data to be processed into the subject-object division model for feature division to obtain the subject feature vector, the object feature vector and the shared feature vector, wherein the shared feature vector is the part of the global feature vector other than the subject feature vector and the object feature vector. Since the shared feature vector may contain information representing the subject, information representing the object or information representing the subject-object relationship, the shared feature vector is a feature vector that helps to enhance the information representation, and this part of the feature vector is not discarded in the subsequent processing; further, by inputting the subject feature vector, the object feature vector and the shared feature vector into the subject-object filtering model, the subject feature vector and the shared feature vector can be feature fused, so that the expression of the subject information in the subject feature vector is enhanced and the expression of the object information is filtered out through the shared feature vector, thereby improving the representation strength of the subject information by the subject feature vector after filtering; the object feature vector and the shared feature vector can be feature fused, so that the expression of the object information in the object feature vector is enhanced and the expression of the subject information is filtered out through the shared feature vector, thereby improving the representation strength of the object feature vector for the object information after filtering. Based on the filtered subject feature vector and the filtered object feature vector with enhanced representation strength, the probability of subject determination error, object determination error or relationship matching error when obtaining relationship triples can be reduced, thereby improving the accuracy of extracting relationship triples.

[0104] In a possible implementation, the method also includes: inputting the text data to be processed into an encoder for encoding the input information as a feature vector to obtain a text feature vector corresponding to the text data; fusing the text feature vector with multiple pre-encoded relationship feature vectors using a multi-head attention mechanism to obtain a global feature vector corresponding to the text data, and any relationship feature vector is used to represent a relationship in a relationship triplet.

[0105] Exemplarily, the text feature vector and the relationship feature vector can be fused through a text-relation encoding module to obtain a global feature vector corresponding to the text data. The text-relation encoding module can be understood as a neural network model including at least one encoder.

[0106] Figure 5 This is a schematic diagram of the structure of the text-relation encoding module provided in the embodiment of the present application. Figure 5 As shown, the text-relation encoding module may include a bidirectional encoder representation from transformers (BERT) encoder, which can be used to encode the feature vector of the text data to be processed to obtain a text feature vector H corresponding to the text data, where hn Represents the information corresponding to the nth token in the text feature vector.

[0107] For example, the text data can be segmented into i Input into the BERT encoder and get the corresponding feature vector representation h i , the calculation process formula satisfies: Among them, t i Represents the i-th token obtained after word segmentation of text data; Represents the output information of the first layer of the BERT encoder for the i-th token; TRM represents transformers including multi-head attention mechanism, fully connected layer and normalization layer. Represents the output information of the last layer of the BERT encoder for the i-th token; W represents the output information of the i-1 layer in the BERT encoder for the i-th token; e represents the weight matrix of the BERT encoder; b e represents the bias matrix of the BERT encoder; TRM, W e and b e Both can be pre-trained on large-scale corpora.

[0108] By combining the output information of each token in the last layer of the BERT encoder, we can get the text feature vector H corresponding to the text data, which can be expressed as Among them, n represents the length of the input sequence after slicing, which can be understood as the number of tokens obtained after word segmentation is n, and n is a positive integer. It can be understood as the output information of the last layer in the BERT encoder for the nth token.

[0109] In order to mine the association between each token and relationship in the text data, the obtained text feature vector H and the relationship feature vector R can be fused by a multi-head attention mechanism to enhance the representation of the text feature vector.

[0110] like Figure 5 As shown, the text-relation encoding module may also include a relation encoder, which may be used to perform feature vector encoding on the relation to obtain a relation feature vector R corresponding to the relation, where R m Represents the relation feature vector corresponding to the mth relation, where m is a positive integer. The relation encoder can be a BERT encoder or an encoder of other model structures.

[0111] For example, a relationship set including multiple relationships may be preset, such as {like, dislike, good, bad, willing to save money, refuse to handle...}. The relationship encoder encodes the feature vector of each relationship in the relationship set to obtain multiple pre-encoded relationship feature vectors.

[0112] For example, the text feature vector can be fused with multiple pre-encoded relationship feature vectors through a multi-head attention mechanism to obtain a global feature vector corresponding to the text data. Figure 5 As shown in the figure, in the text-relation encoding module, the pre-defined relationship is encoded by the relation encoder to obtain m relation feature vectors R. The text data to be processed is encoded by the feature vector to obtain the text feature vector H. Through the network structure of the multi-head attention mechanism in the text-relation encoding module, the multi-head attention fusion operation is performed on H and R to obtain the global feature vector H of the text data. R , where h Rn Denotes the global eigenvector H R The nth element in can also be understood as the global eigenvector H R The information corresponding to the nth token in .

[0113] For example, when the multi-head attention mechanism is fused on the text feature vector H and the relationship feature vector R, the following formula is satisfied:

[0114] Q=W q H;

[0115] K=W k H;

[0116] V=W v H;

[0117]

[0118] H R =K·(head1, head2,..., head L )

[0119] Where Q represents the query matrix; W q represents the weight matrix of the query matrix; K represents the key matrix; W k represents the weight matrix of the key matrix; V represents the value matrix; W v The weight matrix representing the value matrix; head L Represents the output information of the Lth attention head; d k represents the dimension of the bond matrix; W q , W k and W v It can be obtained through model training.

[0120] In the multi-head attention mechanism, the input data can be linearly transformed into multiple Q, K, V vector groups, each of which is called an attention head. These attention heads perform attention calculations independently to capture different features or patterns. Finally, the output information of each attention head is connected together and linearly transformed to generate the final output.

[0121] Based on this, the multi-head attention mechanism can enable the text-relation encoding module to focus on different parts of the text feature vector at the same time, so that the information representation of the global feature vector obtained is more biased towards the information representation of multiple pre-encoded relationship feature vectors than the information representation of the text feature vector, thereby improving the representation strength of the global feature vector in the dimension of the corresponding relationship of each relationship feature vector, and obtaining more accurate results when using the global feature vector for subject-object division and filtering. Therefore, the text feature vector of the text data and the relationship feature vector of the relationship in the triplet can be integrated through the multi-head attention mechanism of the text-relation encoding module to improve the accuracy of the model in predicting triples.

[0122] In one possible implementation, relation triples are used to perform public opinion analysis on a bank, and the method further includes: obtaining text data associated with the bank; performing data preprocessing on the text data associated with the bank to obtain preprocessed text data, wherein the data preprocessing includes at least one of character correction and text length adjustment; and determining the preprocessed text data as text data to be processed.

[0123] Exemplarily, the text data associated with a bank may include text data related to information about at least one bank, for example, text data in any natural language describing a bank's products, branches, services and / or events. The text data may be, for example, comment text or complaint suggestion text.

[0124] Figure 6 Schematic diagram of the text data processing method provided in the embodiment of the present application Figure 2 ,like Figure 6 As shown, if Internet nodes allow, text data associated with the bank can be obtained through crawler technology of programming languages ​​(such as Python), and the text data can be preprocessed, and the data preprocessing can achieve data cleaning.

[0125] Exemplarily, when acquiring text data associated with a bank, text content of user reviews, user complaints, user questionnaires, etc. may also be collected through the bank's application program and determined as text data associated with the bank.

[0126] In some scenarios, the text data associated with the bank may have inconsistent characters, garbled characters and / or wrong codes, etc. The data preprocessing through character correction can unify the characters and correct the garbled characters and / or wrong codes to improve the availability of the text data. For another example, when extracting the global feature vector, there may be a limit on the number of input characters. If the character length of the text data exceeds the limit on the number of input characters, important information may be missing or global feature extraction may not be possible. Therefore, data preprocessing of the text data associated with the bank to adjust the text length can avoid the situation where important information is missing or global feature extraction cannot be performed due to the character length of the text data exceeding the limit on the number of input characters, thereby improving the applicability of the method. When performing data preprocessing, it can be done through manual processing or data preprocessing by a machine learning model, which is not limited in the embodiments of the present application.

[0127] In addition, during the model training phase of the text-relation encoding module, the subject-object encoding module and / or the triple extraction module, data annotation may also be performed on the text data samples after data preprocessing to annotate the real subjects and objects therein.

[0128] like Figure 6 As shown, after data preprocessing, the obtained preprocessed text data can be determined as the text data to be processed, and input into the text-relation coding module for global feature vector extraction to obtain the global feature vector corresponding to the text data. Afterwards, the global feature vector can be input into the subject-object coding module, and the subject feature vector, object feature vector and shared feature vector can be divided through the subject-object division model. Based on the subject-object filtering model, the subject feature vector and the object feature vector are filtered respectively to obtain the filtered subject feature vector and the filtered object feature vector. The filtered subject feature vector and the filtered object feature vector are input into the triple extraction module respectively, and the relation triples in the text data associated with the bank can be obtained. According to the obtained relation triples, the public opinion analysis of the bank can be carried out to obtain the public opinion analysis results such as the public opinion analysis information.

[0129] For example, after extracting relational triples from a large amount of text data associated with a bank, several relational triples can be obtained. The subjects in these relational triples can be users of the bank or potential users of the bank, etc. The objects in these relational triples can be the brand of the bank, the bank's financial products, the bank's service personnel or the bank's hardware facilities, etc. The relations in these relational triples can be the subject's actions, emotions, attitudes or intentions towards the objects, etc. According to the classification statistics of the subjects, objects and / or relations in these triples, the public opinion of users or potential users on the bank can be obtained, that is, the public opinion analysis results can be obtained. The bank can adjust the products, services or hardware facilities it provides according to the public opinion analysis results to improve the user experience.

[0130] In this embodiment, the goal of processing natural language text data to obtain triples therein and analyzing bank public opinion based on the obtained triples is achieved. Based on this, the limitations of public opinion analysis based on sentiment analysis and other methods in the prior art can be overcome, and a more efficient, more accurate and more general public opinion analysis method is provided, which is particularly suitable for processing text data in professional fields such as bank public opinion and financial reviews. By extracting triples with higher accuracy, positive and negative emotions can be obtained from the extracted relationships in a more fine-grained manner, and the subject object and the object object to which the emotions are directed can be clearly identified. In particular, when different emotional expressions are made for multiple objects in the text data, based on relatively accurate triple extraction, the emotional expression for a specific object can be accurately identified.

[0131] For example, among the related methods of public opinion analysis, there are also methods based on topic analysis. For example, the words and sentences used in public comments are usually more casual, and the content is also more scattered. The topic of the comments is mined in a large amount of comment corpus, and the mainstream latent dirichlet allocation (LDA) model is used for mining. The basic idea of ​​LDA is to calculate the probability distribution in the expected vocabulary through probability distribution, thereby reflecting the theme of the comment text. The calculation process first randomly assigns a topic to each word, uses the Gibbs sampling method for sampling, and calculates the probability distribution of each word in each topic, then reassigns a topic to the current word, repeats the calculation until the Gibbs sampling converges, and finally determines the distribution of the topic and probability of each comment text, and outputs the characteristic words of each topic.

[0132] In this method, there are limitations in semantic understanding. Topic analysis mainly relies on word frequency and co-occurrence statistics, which may not be enough to accurately understand complex semantic relationships and emotions. For example, it may be difficult to distinguish the subtle differences between synonyms or near-synonyms in different contexts. At the same time, emotional details are ignored. Topic analysis focuses more on identifying and classifying topics rather than deeply analyzing the details and changes of emotions under each topic. In contrast, triple extraction combined with sentiment analysis can provide more specific emotional tendencies and attitudes. In addition, this method is more fuzzy about topics. In some cases, topic analysis may produce results with blurred topic boundaries, especially when there are multiple related but not identical topics in the text data. This may lead to reduced interpretability of the analysis results. In addition, this method relies on a large amount of text data. Topic analysis methods, especially LDA, usually require a large amount of text data to generate a stable topic model. For cases with small data volumes or short text fragments, the accuracy and stability of topic analysis may be affected.

[0133] Exemplarily, when the method provided by the embodiment of the present application is applied to the scenario of public opinion analysis, the relationship between entities can be more accurately identified. The triple extraction method can accurately identify the relationship between entities in the text, including the subject, predicate and object, which is very critical for understanding the public opinion attitude and emotional orientation of a specific entity. It goes beyond the generalized topic identification of topic analysis and provides more specific relationship details. Through the method of the embodiment of the present application, fine-grained combination of emotions and topics can also be performed. Combined with sentiment analysis, the triple extraction method can provide fine-grained analysis of sentiment tendencies, not only identifying positive or negative emotions, but also clarifying the entities and events to which the emotions point. This is more in-depth than simple topic or sentiment analysis and helps to understand the specific reasons behind complex emotions. Furthermore, the method of the embodiment of the present application can also identify context sensitivity. Triple extraction can better handle context-sensitive information, and accurately capture the specific context in the text through the identification of entities and relationships, which is crucial for understanding complex emotional expressions such as sarcasm and irony. In addition, the flexibility and customization of this method are good. Triplet extraction can be customized according to specific analysis needs, for example, focusing on the analysis of specific entities or events, which makes public opinion analysis more focused and efficient.

[0134] In addition, the method provided by the above embodiment can improve the accuracy of determining triples in text data. In addition to being applicable to public opinion analysis of banks, the above models or modules can also be extended to perform triple extraction and / or public opinion analysis on text data in other scenarios by inputting different training data.

[0135] Figure 7 A structural diagram of a text data processing device provided in an embodiment of the present application is shown in FIG. Figure 7As shown, an embodiment of the present application provides a text data processing device, the device comprising:

[0136] The first processing module 701 is used to input the global feature vector corresponding to the text data to be processed into the subject-object division model to obtain the subject feature vector, the object feature vector and the shared feature vector; wherein the subject-object division model is a network model used to distinguish the subject and the object in the relation triple of the input information, the subject feature vector is the part corresponding to the subject in the global feature vector, the object feature vector is the part corresponding to the object in the global feature vector, and the shared feature vector is the part of the global feature vector other than the subject feature vector and the object feature vector;

[0137] The first processing module 701 is further used to input the subject feature vector, the object feature vector and the shared feature vector into the subject-object filtering model, perform feature fusion on the subject feature vector and the shared feature vector to obtain the filtered subject feature vector, and perform feature fusion on the object feature vector and the shared feature vector to obtain the filtered object feature vector; wherein the subject-object filtering model is a network model used to perform feature fusion on the input information;

[0138] The second processing module 702 is used to obtain the relation triples in the text data according to the filtered subject feature vector and the filtered object feature vector.

[0139] In a possible implementation, the subject-object division model includes a subject gate and an object gate, the subject gate is a sub-model in the subject-object division model for determining the subject feature vector through a gating mechanism, and the object gate is a sub-model in the subject-object division model for determining the object feature vector through a gating mechanism; the first processing module 701 is specifically used to: input the global feature vector corresponding to the text data to be processed into the subject gate to obtain the subject feature vector, and input the global feature vector corresponding to the text data to be processed into the object gate to obtain the object feature vector; determine the part of the global feature vector other than the subject feature vector and the object feature vector as a shared feature vector.

[0140] In one possible implementation, the main gate satisfies the formula when processing the global feature vector: Among them, x t represents the tth element in the global eigenvector, h t-1 represents the hidden state at the previous moment, represents the weight matrix of the main gate, The weight matrix representing the hidden state of the main gate, represents the bias vector of the main gate, cummax represents the cumulative sum operation, Represents the feature vector corresponding to the tth element in the subject feature vector; the object gate satisfies the formula when processing the global feature vector: in, represents the weight matrix of the object gate, The weight matrix representing the hidden state of the object gate, represents the bias vector of the object gate, Represents the feature vector corresponding to the tth element in the object feature vector.

[0141] In one possible implementation, the first processing module 701 is specifically used for: adding the subject feature vector and the shared feature vector and then performing data processing using a hyperbolic tangent activation function to obtain a filtered subject feature vector; the first processing module 701 is specifically used for: adding the object feature vector and the shared feature vector and then performing data processing using a hyperbolic tangent activation function to obtain a filtered subject feature vector.

[0142] In one possible implementation, the second processing module 702 is specifically used to: process the shared feature vector through data processing of the hyperbolic tangent activation function to obtain a processed shared feature vector; input the feature vector obtained by connecting the filtered subject feature vector and the processed shared feature vector into a subject extraction model to obtain the subject in the text data, and the subject extraction model is a network model for decoding input information; input the feature vector obtained by connecting the filtered object feature vector and the processed shared feature vector into an object extraction model to obtain the object in the text data, and the object extraction model is a network model for decoding input information; based on the subject and the object, obtain the relationship triples in the text data.

[0143] In a possible implementation, the first processing module 701 is also used to: input the text data to be processed into an encoder used to perform feature vector encoding on the input information to obtain a text feature vector corresponding to the text data; fuse the text feature vector with multiple pre-encoded relationship feature vectors through a multi-head attention mechanism to obtain a global feature vector corresponding to the text data, and any relationship feature vector is used to represent a relationship in a relationship triplet.

[0144] In one possible implementation, the relation triples are used to perform public opinion analysis on a bank, and the first processing module 701 is also used to: obtain text data associated with the bank; perform data preprocessing on the text data associated with the bank to obtain preprocessed text data, wherein the data preprocessing includes at least one of character correction and text length adjustment; and determine the preprocessed text data as text data to be processed.

[0145] The text data processing device provided in the embodiment of the present application can be used to execute the technical solution of the text data processing method in any of the above embodiments of the present application. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0146] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the electronic device of this embodiment may include: at least one processor 801; and a memory 802 communicatively connected to the at least one processor; wherein the memory 802 stores instructions executable by the at least one processor 801, and the instructions are executed by the at least one processor 801 so that the electronic device executes a method as in any of the above embodiments.

[0147] Optionally, the memory 802 may be independent or integrated with the processor 801 .

[0148] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the aforementioned embodiments and will not be described in detail here.

[0149] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method described in any of the above embodiments is implemented.

[0150] An embodiment of the present application further provides a computer program product, including a computer program, which implements the method described in any of the above embodiments when executed by a processor.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0152] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application.

[0153] It should be understood that the above-mentioned processor can be a processing unit (Central Processing Unit, CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application-specific integrated circuits (Application Specific Integrated Circuit, ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The memory may include random access memory (Random Access Memory, RAM), and may also include non-volatile memory (Non-Volatile Memory, NVM), such as at least one disk storage, and can also be a USB flash drive, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0154] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0155] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in a dedicated integrated circuit. Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0156] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0157] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0158] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0159] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

[0160] Those skilled in the art will readily come up with other implementations of the embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modifications, uses or adaptations of the embodiments of the present application, which follow the general principles of the embodiments of the present application and include common knowledge or customary technical means in the art that are not disclosed in the embodiments of the present application.

[0161] It should be understood that the embodiments of the present application are not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the embodiments of the present application is limited only by the appended claims.

Claims

1. A text data processing method, characterized in that: The method comprises: Inputting the global feature vector corresponding to the text data to be processed into the subject-object division model to obtain a subject feature vector, an object feature vector and a shared feature vector; wherein the subject-object division model is a network model used to distinguish the subject and the object in the relation triple of the input information, the subject feature vector is the part of the global feature vector corresponding to the subject, the object feature vector is the part of the global feature vector corresponding to the object, and the shared feature vector is the part of the global feature vector other than the subject feature vector and the object feature vector; Inputting the subject feature vector, the object feature vector and the shared feature vector into a subject-object filtering model, performing feature fusion on the subject feature vector and the shared feature vector to obtain a filtered subject feature vector, and performing feature fusion on the object feature vector and the shared feature vector to obtain a filtered object feature vector; wherein the subject-object filtering model is a network model for performing feature fusion on input information; According to the filtered subject feature vector and the filtered object feature vector, the relation triples in the text data are obtained.

2. The method according to claim 1, characterized in that The subject-object division model includes a subject gate and an object gate, wherein the subject gate is a sub-model in the subject-object division model used to determine the subject feature vector through a gating mechanism, and the object gate is a sub-model in the subject-object division model used to determine the object feature vector through a gating mechanism; The global feature vector corresponding to the text data to be processed is input into the subject-object division model to obtain the subject feature vector, the object feature vector and the shared feature vector, including: Inputting the global feature vector corresponding to the text data to be processed into a subject gate to obtain a subject feature vector, and inputting the global feature vector corresponding to the text data to be processed into an object gate to obtain an object feature vector; A portion of the global feature vector excluding the subject feature vector and the object feature vector is determined as the shared feature vector.

3. The method according to claim 2, characterized in that The main gate satisfies the formula when processing the global feature vector: Among them, x t represents the tth element in the global feature vector, h t-1 represents the hidden state at the previous moment, represents the weight matrix of the subject gate, represents the weight matrix of the hidden state of the subject gate, represents the bias vector of the main gate, cummax represents the cumulative sum operation, represents the feature vector corresponding to the tth element in the subject feature vector; The object gate satisfies the formula when processing the global feature vector: in, represents the weight matrix of the object gate, The weight matrix representing the hidden state of the object gate, represents the bias vector of the object gate, Represents the feature vector corresponding to the tth element in the object feature vector.

4. The method according to claim 1, characterized in that: The step of fusing the subject feature vector and the shared feature vector to obtain the filtered subject feature vector includes: The main feature vector and the shared feature vector are added together and then subjected to data processing using a hyperbolic tangent activation function to obtain the filtered main feature vector; The step of performing feature fusion on the object feature vector and the shared feature vector to obtain a filtered object feature vector includes: The object feature vector and the shared feature vector are added together and then subjected to data processing using a hyperbolic tangent activation function to obtain the filtered subject feature vector.

5. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining the relation triples in the text data according to the filtered subject feature vector and the filtered object feature vector comprises: Processing the shared feature vector by a hyperbolic tangent activation function to obtain a processed shared feature vector; Inputting a feature vector obtained by connecting the filtered subject feature vector and the processed shared feature vector into a subject extraction model to obtain the subject in the text data, wherein the subject extraction model is a network model for decoding input information; Inputting the feature vector obtained by connecting the filtered object feature vector and the processed shared feature vector into an object extraction model to obtain the object in the text data, wherein the object extraction model is a network model for decoding input information; Based on the subject and the object, a relation triple in the text data is obtained.

6. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Inputting the text data to be processed into an encoder for encoding input information into a feature vector to obtain a text feature vector corresponding to the text data; The text feature vector is fused with multiple pre-encoded relation feature vectors through a multi-head attention mechanism to obtain a global feature vector corresponding to the text data, and any of the relation feature vectors is used to represent a relation in a relation triple.

7. The method according to any one of claims 1 to 4, characterized in that: The relationship triples are used to perform public opinion analysis on the bank, and the method further includes: obtaining text data associated with the bank; Performing data preprocessing on the text data associated with the bank to obtain preprocessed text data, wherein the data preprocessing includes at least one of character correction and text length adjustment; The preprocessed text data is determined as the text data to be processed.

8. An electronic device, characterized in that: include: Memory and processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images