Text analysis method and device and related product

Abstract

The invention provides a text analysis method and device and a related product, and relates to the technical field of natural language processing. The method comprises the steps of obtaining a to-be-analyzed text and an embedded code corresponding to the to-be-analyzed text; performing self-attention calculation on the embedded code corresponding to the to-be-analyzed text to obtain a self-attention weight of each word in the to-be-analyzed text; performing abstract semantic analysis on the to-be-analyzed text to obtain an abstract semantic representation map of the to-be-analyzed text; mapping a node relationship in the abstract semantic representation map into a vector to obtain a map relationship embedded code; enhancing the self-attention weight of each word in the to-be-analyzed text through the graph relation embedded codes to obtain the enhanced self-attention weight; and obtaining an enhanced text code of the to-be-analyzed text based on the enhanced self-attention weight and the embedded code of each word in the to-be-analyzed text so as to perform text analysis based on the enhanced text code. According to the embodiment of the invention, the accuracy of task analysis can be improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of natural language processing, and in particular to a text analysis method, device, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] In the task of fine-grained sentiment analysis, the system needs to extract fine-grained sentiment elements such as evaluation objects and evaluation content from the text. The most common method is to judge the boundaries of elements in the text based on the discriminant model, or to directly generate sentiment element content based on the generative model. Both rely on the multi-head self-attention structure in the pre-trained model to inject contextual information into the text token encoding to improve the accuracy of the analysis task.

[0004] Although the above-mentioned fine-grained sentiment analysis method based on pre-training models can obtain token encodings containing rich semantic context information, when the length of the input text exceeds a certain limit, too much context information will cause distraction, ultimately leading to poor analysis task results. Summary of the invention

[0005] The purpose of the present disclosure is to provide a text analysis method, device, electronic device, computer-readable storage medium and computer program product, which can improve the accuracy of analysis tasks.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] The present disclosure provides a text analysis method, including: obtaining a text to be analyzed and an embedded code corresponding to the text to be analyzed; performing self-attention calculation on the embedded code corresponding to the text to be analyzed to obtain the self-attention weight of each word in the text to be analyzed; performing abstract semantic analysis on the text to be analyzed to obtain an abstract semantic representation graph of the text to be analyzed, wherein the nodes in the abstract semantic representation graph correspond to the words in the text to be analyzed one-to-one; mapping the node relationship in the abstract semantic representation graph into a vector to obtain a graph relationship embedding code; enhancing the self-attention weight of each word in the text to be analyzed by the graph relationship embedding code to obtain an enhanced self-attention weight; obtaining an enhanced text code of the text to be analyzed based on the enhanced self-attention weight and the embedded code of each word in the text to be analyzed, so as to perform text analysis based on the enhanced text code.

[0008] In some embodiments, the self-attention weights of each word in the text to be analyzed are enhanced by the graph relationship embedding code to obtain the enhanced self-attention weight, including: performing outer product calculation on the words with association relationship in the text to be analyzed to obtain the text relationship embedding code; fusing the graph relationship embedding code and the text relationship embedding code to obtain the fused relationship code; enhancing the self-attention weights of each word in the text to be analyzed by the fused relationship code to obtain the enhanced self-attention weight.

[0009] In some embodiments, the self-attention weights of each word in the text to be analyzed are enhanced by the graph relationship embedding code to obtain the enhanced self-attention weights, including: superimposing the graph relationship embedding code on the self-attention weights of each word in the text to be analyzed to obtain the enhanced self-attention weights.

[0010] In some embodiments, the text to be analyzed includes a first word, a second word and a third word; the graph relationship embedding code includes a first graph relationship embedding code for describing the relationship between the first word and the second word and a second graph relationship embedding code for describing the relationship between the first word and the third word; the self-attention weight corresponding to the first word includes a first self-attention weight for describing the influence of the second ordered word on the first word and a second self-attention weight for describing the influence of the third word on the first word; wherein, superimposing the graph relationship embedding code on the self-attention weights of each word in the text to be analyzed to obtain the enhanced self-attention weight includes: superimposing the first graph relationship embedding code on the first self-attention weight to obtain a first enhanced self-attention weight, and the enhanced self-attention weight includes the first enhanced self-attention weight and the second enhanced self-attention weight; superimposing the second graph relationship embedding code on the second self-attention weight to obtain the second enhanced self-attention weight.

[0011] In some embodiments, the self-attention weights of each word in the text to be analyzed are enhanced by the graph relationship embedding coding to obtain the enhanced self-attention weights, including: performing relationship propagation according to the graph relationship embedding coding of the text to be analyzed, and determining the propagation relationship embedding coding between each pair of words in the text to be analyzed; enhancing the self-attention weights of each word in the text to be analyzed by the propagation relationship embedding coding between each pair of words in the text to be analyzed to obtain the enhanced self-attention weights.

[0012] In some embodiments, relationship propagation is performed according to the graph relationship embedding code of the text to be analyzed to determine the propagation relationship embedding code between each pair of words in the text to be analyzed, including: performing relationship propagation according to the graph relationship embedding code of the text to be analyzed to determine the propagation relationship embedding code corresponding to the first ordered word pair in the text to be analyzed; wherein the first ordered word pair includes two words with known relationship embedding codes; determining the second ordered word pair in the text to be analyzed, wherein the relationship embedding code between the second ordered word pair is unknown; determining the first target word pair and the second target word pair according to the middle word between the second ordered word pair, wherein the first target word pair is composed of the first word of the second ordered word pair and the middle word, and the second target word pair is composed of the middle word and the last word of the second ordered word pair; the first ordered word pair includes the first target word pair and the second target word pair; determining the preliminary relationship embedding code corresponding to the middle word according to the relationship embedding code of the first target word pair and the second target word pair; summing the preliminary relationship embedding codes corresponding to the middle words to determine the propagation relationship embedding code corresponding to the second ordered word pair.

[0013] In some embodiments, the preliminary relation embedding code corresponding to the intermediate word is determined based on the relation embedding code of the first target word pair and the second target word pair, including: determining the starting point relation embedding code of the first target word pair based on the propagation relation embedding code of the first target word pair, wherein the starting point relation embedding code of the first target word pair is used to represent the corresponding features when the first target word pair is used as the starting point word pair; determining the end point relation embedding code of the second target word pair based on the propagation relation embedding code of the second target word pair, wherein the end point relation embedding code of the second target word is used to represent the corresponding features when the second target word pair is used as the end point word; multiplying the starting point relation embedding code of the first target word pair and the end point relation embedding code of the second target word pair to obtain the preliminary relation embedding code of the intermediate word.

[0014] In some embodiments, according to the propagation relationship embedding code of the first target word pair, the starting point relationship embedding code of the first target word pair is determined, including: determining the update ratio and the starting point relationship embedding code of the first target word pair according to the propagation relationship embedding code of the first target word pair; wherein the update ratio of the first target word pair is used to determine the contribution of the first target word pair in the parameter propagation process; updating the starting point relationship embedding code of the first target word pair by the update ratio of the first target word pair, wherein the larger the update ratio of the first target word pair, the more important the starting point relationship embedding code and the end point relationship embedding code corresponding to the first target word; wherein, according to the propagation relationship embedding code of the second target word pair, determining the end point relationship embedding code of the second target word pair, including: determining the update ratio and the starting point relationship embedding code of the second target word pair according to the propagation relationship embedding code of the second target word pair; updating the starting point relationship embedding code of the second target word pair by the update ratio of the second target word pair, wherein the larger the update ratio of the second target word pair, the more important the starting point relationship embedding code and the end point relationship embedding code corresponding to the second target word.

[0015] In some embodiments, the preliminary relation embedding codes corresponding to each intermediate word are summed to determine the propagation relation embedding code corresponding to the second ordered word pair, including: summing the preliminary relation embedding codes corresponding to each intermediate word to determine the intermediate relation embedding code corresponding to the second ordered word pair; determining the output ratio corresponding to the second ordered word pair according to the intermediate relation embedding code corresponding to the second ordered word pair, the output ratio being used to determine the output ratio of the second ordered word pair when it is output; performing gate processing on the intermediate relation embedding code corresponding to the second ordered word pair according to the output ratio of the second ordered word pair when it is output, to obtain the propagation relation embedding code of the second ordered word pair.

[0016] The disclosed embodiment provides a text analysis device, including: a text acquisition module, a self-attention weight determination module, an abstract semantic representation graph determination module, a graph relationship embedding coding determination module, an enhanced self-attention weight determination module, and an enhanced text coding determination module.

[0017] Among them, the text acquisition module is used to obtain the text to be analyzed and the embedded code corresponding to the text to be analyzed; the self-attention weight determination module can be used to perform self-attention calculation on the embedded code corresponding to the text to be analyzed, and obtain the self-attention weight of each word in the text to be analyzed; the abstract semantic representation graph determination module can be used to perform abstract semantic analysis on the text to be analyzed, and obtain the abstract semantic representation graph of the text to be analyzed, wherein the nodes in the abstract semantic representation graph correspond one-to-one to the words in the text to be analyzed; the graph relationship embedding code determination module can be used to map the node relationship in the abstract semantic representation graph into a vector to obtain the graph relationship embedding code; the enhanced self-attention weight determination module can be used to enhance the self-attention weight of each word in the text to be analyzed through the graph relationship embedding code to obtain the enhanced self-attention weight; the enhanced text code determination module can be used to obtain the enhanced text code of the text to be analyzed based on the enhanced self-attention weight and the embedded code of each word in the text to be analyzed, so as to perform text analysis based on the enhanced text code.

[0018] An embodiment of the present disclosure provides an electronic device, which includes: a memory and a processor; the memory is used to store computer program instructions; the processor calls the computer program instructions stored in the memory to implement any of the above-mentioned text analysis methods.

[0019] The embodiment of the present disclosure provides a computer-readable storage medium on which computer program instructions are stored to implement the text analysis method as described in any one of the above items.

[0020] The present disclosure provides a computer program product or a computer program, which includes computer program instructions stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium, and a processor executes the computer program instructions to implement the above-mentioned text analysis method.

[0021] The text analysis method, device, electronic device, computer-readable storage medium and computer program product provided by the embodiments of the present disclosure can perform abstract semantic analysis on the text to be analyzed, and then integrate the node relationships between the words after the abstract semantic analysis into the self-attention weights corresponding to each word, thereby improving the attention accuracy and thus improving the accuracy of the analysis task.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0024] Figure 1 A scenario schematic diagram showing a text analysis method or a text analysis device that can be applied to an embodiment of the present disclosure is shown.

[0025] Figure 2 The figure is a flowchart of a text analysis method according to an exemplary embodiment.

[0026] Figure 3 The present invention is a flowchart of a text analysis method according to an exemplary embodiment.

[0027] Figure 4 It is a flowchart of a method for updating self-attention weights according to an exemplary embodiment.

[0028] Figure 5 It is a flowchart of a method for enhancing self-attention weight according to an exemplary embodiment.

[0029] Figure 6 The figure is a flowchart of a relation embedding coding relation propagation method according to an exemplary embodiment.

[0030] Figure 7 The present invention is a flowchart of a method for determining a preliminary relation embedding code corresponding to an intermediate word according to an exemplary embodiment.

[0031] Figure 8 It is a structural diagram corresponding to a text semantic analysis method shown according to an exemplary embodiment.

[0032] Fig. 9 The invention is a block diagram of a text analysis device according to an exemplary embodiment.

[0033] Fig.10 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0034] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0035] Those skilled in the art will appreciate that the embodiments of the present disclosure may be a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0036] The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0037] In the embodiments of the present disclosure, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0038] The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0039] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0040] In the description of the present disclosure, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is only a kind of association relationship describing the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "plurality" means two or more. The words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not limit them to be different; the terms "comprising", "including" and "having" are used to express the meaning of open inclusion and mean that in addition to the listed elements / components / etc., there may be other elements / components / etc.

[0041] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0042] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0043] First, some terms involved in the embodiments of the present disclosure are explained to facilitate understanding by those skilled in the art.

[0044] AMR: Abstract Meaning Representation, abstract semantic representation analysis method. AMR is a new type of sentence semantic representation method that aims to simplify the processing of semantic tasks by extracting key information in sentences (such as entities, relations, etc.). AMR explicitly represents the propositional structure of the sentence, removes information in the text that is irrelevant to the semantic task, and makes important information more prominent, thereby theoretically simplifying the difficulty of the model learning to perform these tasks. The basic structure of AMR is a "single-rooted directed acyclic graph", in which content words are abstracted as concepts as "nodes" on the graph, and function words are abstracted as "edges".

[0045] AMR captures “who is doing what to whom” in a sentence. Each sentence is represented as a rooted, directed, acyclic graph with labels on the edges (each edge represents a semantic relation) and leaves (each leaf node represents a concept).

[0046] The foregoing introduces some terms and concepts involved in the embodiments of the present disclosure. The following introduces the technical features involved in the embodiments of the present disclosure.

[0047] The exemplary embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0048] Figure 1 A scenario schematic diagram showing a text analysis method or a text analysis device that can be applied to an embodiment of the present disclosure is shown.

[0049] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment provided by an exemplary embodiment of the present disclosure.

[0050] like Figure 1 As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0051] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Terminal devices 101, 102, 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, wearable devices, virtual reality devices, smart homes, etc.

[0052] The server 105 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received request data, and feed back the processing results to the terminal device.

[0053] The server can be an independent physical server or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and natural language processing platforms, etc. The present disclosure does not impose any restrictions on this.

[0054] The server 105 may, for example, obtain the text to be analyzed and the embedded code corresponding to the text to be analyzed; the server 105 may, for example, perform self-attention calculation on the embedded code corresponding to the text to be analyzed, and obtain the self-attention weight of each word in the text to be analyzed; the server 105 may, for example, perform abstract semantic analysis on the text to be analyzed, and obtain an abstract semantic representation graph of the text to be analyzed, wherein the nodes in the abstract semantic representation graph correspond one-to-one to the words in the text to be analyzed; the server 105 may, for example, map the node relationships in the abstract semantic representation graph into vectors, and obtain graph relationship embedding codes; the server 105 may, for example, enhance the self-attention weights of each word in the text to be analyzed through graph relationship embedding codes, and obtain enhanced self-attention weights; the server 105 may, for example, obtain an enhanced text code of the text to be analyzed based on the enhanced self-attention weights and the embedded codes of each word in the text to be analyzed, so as to perform text analysis based on the enhanced text code.

[0055] It should be understood that Figure 1 The number of terminal devices, networks and servers in the figure is only for illustration. The server 105 may be a physical server or may be composed of multiple servers. According to actual needs, it may have any number of terminal devices, networks and servers.

[0056] Under the above system architecture, a text analysis method is provided in an embodiment of the present disclosure, and the method can be executed by any electronic device with computing and processing capabilities.

[0057] Figure 2 is a flowchart of a text analysis method according to an exemplary embodiment. The method provided by the embodiment of the present disclosure can be executed by any electronic device with computing processing capabilities. For example, the method can be executed by the above Figure 1 The server or terminal device in the embodiment may be executed, or the server and the terminal device may be executed together. In the following embodiments, the server is used as the execution subject for example, but the present disclosure is not limited to this.

[0058] Reference Figure 2 The text analysis method provided by the embodiment of the present disclosure may include the following steps.

[0059] Step S202, obtaining the text to be analyzed and the embedded code corresponding to the text to be analyzed.

[0060] In some embodiments, the text to be analyzed may be any text that needs to be analyzed.

[0061] In some embodiments, the text to be analyzed may be segmented, and then the segmented text to be analyzed may be input into an embedding model to obtain an embedding vector for each segmented word.

[0062] For example, the embedded code of the text to be analyzed (or the corresponding words in the text to be analyzed) can be obtained through a pre-trained language model (PLM).

[0063] Step S204, performing self-attention calculation on the embedded code corresponding to the text to be analyzed, and obtaining the self-attention weight of each word in the text to be analyzed.

[0064] In some embodiments, a self-attention mechanism can be used to perform self-attention calculation on the embedded code corresponding to the text to be analyzed to obtain the self-attention weight of each word in the text to be analyzed.

[0065] The self-attention weight in the self-attention mechanism is a key calculation result that reflects the degree of correlation or importance between each element in the input sequence.

[0066] In the self-attention mechanism, each input element (such as a word in a text or a frame in a voice) generates a corresponding query, key, and value vector. These vectors are usually obtained by inputting the embedding vector of the input element into a linear transformation layer. Then, the similarity (such as the dot product) between the query vector and the key vectors of all elements in the sequence is calculated to obtain a score that reflects the similarity between the query vector and each key vector, that is, the degree of association between each element and the current element. Next, these scores are normalized by a Softmax function to obtain the attention weights of each element (such as the words in the text to be analyzed in this application). These weights indicate how much attention should be given to other elements in the sequence when understanding the current element. In other words, the self-attention weights measure the importance of different elements in the sequence to the current element.

[0067] Step S206, performing abstract semantic analysis on the text to be analyzed to obtain an abstract semantic representation graph of the text to be analyzed, wherein the nodes in the abstract semantic representation graph correspond one-to-one to the words in the text to be analyzed.

[0068] Abstract semantic representation (AMR) extracts semantic entities in sentences and constructs graphs of entity relationships, representing the deep meaning of natural language sentences in a structured way. Applying abstract semantic representation to feature relationship recognition has become a common information enhancement method. Related research includes using abstract semantic representation to guide the recognition of target entities in knowledge graph queries, or attaching abstract semantic representation to the original text representation vector for event extraction.

[0069] In some embodiments, abstract semantic analysis technology can be used to perform abstract semantic analysis on the text to be analyzed to obtain an abstract semantic representation graph of the text to be analyzed, wherein the nodes in the abstract semantic representation graph correspond one-to-one to the words in the text to be analyzed, and wherein two interconnected nodes in the abstract semantic representation graph represent that the words or concepts corresponding to the two nodes have an association relationship.

[0070] For example, in “I want to go to XX shopping mall to buy apples”, “I” and “mall” are associated, and “mall” and “apples” are associated. Then in the abstract semantic representation graph, “I” and “mall” are connected, “mall” and “apples” are connected, and so on.

[0071] In some embodiments, after the nodes in the graph are matched one by one with the words in the text, it is possible to clearly know which node in the graph the text word corresponds to, that is, once a node is determined, the text word corresponding to the node can be known.

[0072] Step S208, mapping the node relationships in the abstract semantic representation graph into vectors to obtain graph relationship embedding codes.

[0073] In some embodiments, the node relationships in the abstract semantic representation graph can be mapped into vectors to obtain graph relationship embedding codes.

[0074] In some embodiments, the relationship names in the abstract semantic representation (AMR) graph can be converted into text form. For example, the node relationship "is part of" can be represented as "part of" or "is part", and it is regarded as a sentence and put into the PLM to obtain the sentence vector (such as the vector corresponding to [CLS], or the average of all word vectors).

[0075] Step S210, the self-attention weight of each word in the text to be analyzed is enhanced by graph relationship embedding coding to obtain the enhanced self-attention weight.

[0076] In some embodiments, the graph relationship embedding code can be superimposed on the self-attention weight of each word in the text to be analyzed to obtain the enhanced self-attention weight.

[0077] In some embodiments, through self-attention calculation, the self-attention weights corresponding to each word in the text to be analyzed can be obtained. Each word will correspond to multiple self-attention weights, one of which can be used to measure the influence of other words on the current word.

[0078] For example, assuming that the text to be analyzed includes three words A, B, and C, then A will correspond to at least two self-attention weights A1 and A2, where A1 is used to measure the influence of word B on word A, and A2 is used to measure the influence of word C on word A.

[0079] In some embodiments, the above-mentioned word A and word B may have an association relationship, so there may be a corresponding graph relationship embedding code between A and B. Then the graph relationship embedding code between A and B can be used to enhance the self-attention weight A1 of A.

[0080] For specific superposition methods, you can also refer to Figure 4 The embodiment shown is not described in detail in this embodiment.

[0081] Step S212, based on the enhanced self-attention weight and the embedded code of each word in the text to be analyzed, obtain the enhanced text code of the text to be analyzed, so as to perform text analysis based on the enhanced text code.

[0082] The above text analysis may be an analysis of the sentiment (such as positive or negative) of the text.

[0083] The above embodiment can perform abstract semantic analysis on the text to be analyzed, and then integrate the node relationships between words after the abstract semantic analysis into the self-attention weights corresponding to each word, thereby improving the attention accuracy and thus improving the accuracy of the analysis task.

[0084] Figure 3 The present invention is a flowchart of a text analysis method according to an exemplary embodiment.

[0085] Step S302, obtaining the text to be analyzed and the embedded code corresponding to the text to be analyzed.

[0086] Step S304, performing self-attention calculation on the embedded code corresponding to the text to be analyzed, and obtaining the self-attention weight of each word in the text to be analyzed.

[0087] Step S306, performing abstract semantic analysis on the text to be analyzed to obtain an abstract semantic representation graph of the text to be analyzed, wherein the nodes in the abstract semantic representation graph correspond one-to-one to the words in the text to be analyzed.

[0088] Step S308, mapping the node relationships in the abstract semantic representation graph into vectors to obtain graph relationship embedding codes.

[0089] Step S310, performing outer product calculation on the words with association relationship in the text to be analyzed to obtain text relationship embedding code.

[0090] In some embodiments, words with associated relationships (such as relationships in an abstract semantic representation graph) in the text to be analyzed can be determined, and then all words with associated relationships are calculated using outer products to obtain text relationship embedding codes for each associated relationship.

[0091] In some other embodiments, the outer product calculation may be performed on any two words in the text to be analyzed, so as to determine the relationship between any two words in the text to be analyzed.

[0092] Among them, the association relationship between the two words can be determined by calculating the outer product between the two words.

[0093] Step S312, the graph relationship embedding code and the text relationship embedding code are fused to obtain a fused relationship code.

[0094] In some embodiments, the graph relationship embedding code and the text relationship embedding code may be fused accordingly to obtain a fused relationship code.

[0095] For example, assuming that it is known that there is an association relationship between A and B, then the graph relationship embedding code and text relationship embedding code between A and B can be obtained respectively, and then the graph relationship embedding code and text relationship embedding code between A and B are fused to obtain the fused relationship code between A and B.

[0096] Step S314, the self-attention weight of each word in the text to be analyzed is enhanced by the fused relational encoding to obtain the enhanced self-attention weight.

[0097] In some embodiments, the fused relation embedding code can be superimposed on the self-attention weights of each word in the text to be analyzed to obtain enhanced self-attention weights.

[0098] In some embodiments, through self-attention calculation, the self-attention weights corresponding to each word in the text to be analyzed can be obtained. Each word will correspond to multiple self-attention weights, one of which can be used to measure the influence of other words on the current word.

[0099] For example, assuming that the text to be analyzed includes three words A, B, and C, then A will correspond to at least two self-attention weights A1 and A2, where A1 is used to measure the influence of word B on word A, and A2 is used to measure the influence of word C on word A.

[0100] In some embodiments, the above-mentioned word A and word B may have an association relationship, so there may be a corresponding fused relationship code between A and B. Then the fused relationship code between A and B can be used to enhance the self-attention weight A1 in A.

[0101] For example, it is assumed that the text to be analyzed includes a first word, a second word, and a third word.

[0102] In some embodiments, the fused relational code may include a first fused relational code for describing the relationship between the first word and the second word and a second fused relational code for describing the relationship between the first word and the third word.

[0103] In some embodiments, the self-attention weight corresponding to the first word includes a first self-attention weight for describing the influence of the second ordered word on the first word and a second self-attention weight for describing the influence of the third word on the first word. Then, the above self-attention weight update method may include the following steps: embedding the first graph relationship into the first self-attention weight to obtain a third enhanced self-attention weight, and the enhanced self-attention weight includes the third enhanced self-attention weight and the fourth enhanced self-attention weight; embedding the second graph relationship into the second self-attention weight to obtain the fourth enhanced self-attention weight.

[0104] Step S316, based on the enhanced self-attention weight and the embedded code of each word in the text to be analyzed, obtain the enhanced text code of the text to be analyzed, so as to perform text analysis based on the enhanced text code.

[0105] Through the above method, the attention accuracy can be improved by embedding the text relationship encoding and fusing the node relationship between words after abstract semantic analysis into the self-attention weight corresponding to each word, thereby improving the accuracy of the analysis task.

[0106] Figure 4 It is a flowchart of a method for updating self-attention weights according to an exemplary embodiment.

[0107] In some embodiments, the text to be analyzed may include a first word, a second word, and a third word. Below, this embodiment will take the first word, the second word, and the third word as examples to explain how to update the self-attention weight of the word.

[0108] In some embodiments, the graph relationship embedding code may include a first graph relationship embedding code for describing the relationship between the first word and the second word and a second graph relationship embedding code for describing the relationship between the first word and the third word.

[0109] In some embodiments, the self-attention weight corresponding to the first word includes a first self-attention weight for describing the influence of the second ordered word on the first word and a second self-attention weight for describing the influence of the third word on the first word.

[0110] refer to Figure 4, the above self-attention weight updating method may include the following steps.

[0111] Step S402: embedding the first graph relationship into the first self-attention weight to obtain a first enhanced self-attention weight, where the enhanced self-attention weight includes a first enhanced self-attention weight and a second enhanced self-attention weight.

[0112] Step S404: embed the second graph relationship into the second self-attention weight to obtain the second enhanced self-attention weight.

[0113] Through the above method, the self-attention weight of words can be enhanced through graph relationship encoding to improve the accuracy of task analysis.

[0114] Figure 5 It is a flowchart of a method for enhancing self-attention weight according to an exemplary embodiment.

[0115] refer to Figure 5 , the above self-attention weight enhancement method may include the following steps.

[0116] Step S502, performing relationship propagation according to the graph relationship embedding code of the text to be analyzed, and determining the propagation relationship embedding code between each two words in the text to be analyzed.

[0117] In some embodiments, relationship propagation can be performed based on the graph relationship embedding code of the text to be analyzed to determine the propagation relationship embedding code between each pair of words in the text to be analyzed.

[0118] For example, if there is word B between word A and word C, then the propagation relationship embedding code between AC can be determined through the graph relationship embedding code between AB and BC. Similarly, the propagation relationship embedding code between two words in the text to be analyzed can be determined through this propagation method.

[0119] Step S504, the self-attention weight of each word in the text to be analyzed is enhanced by embedding the propagation relationship between each two words in the text to be analyzed to obtain the enhanced self-attention weight.

[0120] In some embodiments, the propagation relationship embedding code between two words in the text to be analyzed can be correspondingly superimposed on the self-attention weight of each word to obtain the enhanced self-attention weight. For example, the propagation relationship embedding code between AC can be superimposed on the self-attention weight of C to A.

[0121] The above method can enhance the representation ability of relation embedding coding, thereby improving the accuracy of text analysis.

[0122] Figure 6The figure is a flowchart of a relation embedding coding relation propagation method according to an exemplary embodiment.

[0123] It should be noted that this embodiment is based on graph relationship embedding coding for relationship propagation, and those skilled in the art can also apply the method corresponding to this embodiment to the propagation process of fused relationship coding, and this application does not limit this. Specifically, those skilled in the art can also replace the graph relationship embedding coding in this embodiment with fused relationship coding for propagation to obtain propagation relationship embedding coding.

[0124] refer to Figure 6 , the above-mentioned relation embedding encoding relation propagation method may include the following steps.

[0125] Step S602, performing relation propagation according to the graph relation embedding code of the text to be analyzed, and determining the propagation relation embedding code corresponding to the first ordered word pair in the text to be analyzed; wherein the first ordered word pair includes two words with known relation embedding codes.

[0126] In some embodiments, a plurality of ordered word pairs may be generated by combining words in the text to be analyzed in pairs, wherein the order may refer to the order in the abstract semantic graph.

[0127] In some embodiments, the ordered word pairs in the text to be analyzed may include first ordered word pairs with known relationship embedding codes and second ordered word pairs with unknown relationship embedding codes.

[0128] Among them, the relation embedding code of the first ordered word pair can be determined according to the graph relation embedding code, and can also be determined in the subsequent relation propagation process.

[0129] Step S604, determining second ordered word pairs in the text to be analyzed, wherein the relationship embedding code between the second ordered word pairs is unknown.

[0130] Step S606, determining the first target word pair and the second target word pair based on the middle word between the second ordered word pair, wherein the first target word pair is composed of the first word and the middle word of the second ordered word pair, and the second target word pair is composed of the middle word and the last word of the second ordered word pair; the first ordered word pair includes the first target word pair and the second target word pair.

[0131] In some embodiments, the relation embedding codes between the first ordered word pairs may be propagated through a relation propagation method to determine the relation embedding codes for the second ordered word pairs.

[0132] The details can be explained through the following steps.

[0133] Assume that at least one middle word may be included between the second ordered word pair, and assume that the at least one middle word includes the first middle word and the second middle word. Then the first target word pair and the second target word pair can be determined according to the middle word (such as the first middle word or the second middle word) between the second ordered word pair, wherein the first target word pair is composed of the first word and the middle word (such as the first middle word or the second middle word) of the second ordered word pair, and the second target word pair is composed of the middle word (such as the first middle word or the second middle word) and the last word of the second ordered word pair; the first ordered word pair includes the first target word pair and the second target word pair, that is, the relational embedding code of the first target word pair and the second target word pair is known. Step S608, determine the preliminary relational embedding code corresponding to the middle word according to the relational embedding code of the first target word pair and the second target word pair.

[0134] In some embodiments, the relation embedding code of the first target word pair can be multiplied with the relation embedding code of the second target word pair to determine the preliminary relation embedding code corresponding to the middle word (such as the first middle word or the second middle word), and the relation embedding code of the first target word pair can be multiplied with the relation embedding code of the second target word pair to determine the preliminary relation embedding code corresponding to the middle word (such as the first middle word or the second middle word). Figure 7 The illustrated embodiment determines the preliminary relational embedding code corresponding to the middle word (such as the first middle word or the second middle word), and the present application does not impose any limitation on this.

[0135] Step S610, summing the preliminary relation embedding codes corresponding to the intermediate words to determine the propagation relation embedding codes corresponding to the second ordered word pair.

[0136] In some embodiments, the preliminary relation embedding codes corresponding to each intermediate word can be determined through steps S602 to S608, and then the preliminary relation embedding codes corresponding to each intermediate word can be summed to determine the propagation relation embedding codes corresponding to the second ordered word pair.

[0137] The above-mentioned relation embedding coding determination process of the second ordered word pair can be illustrated by the following example.

[0138] Assume that the first word of the second ordered word pair is the third word and the last word is the fourth word, and the intermediate words between the third word and the fourth word include the fifth word and the sixth word; wherein, the sum of the bitwise multiplication of the starting point relationship embedding code and the end point relationship embedding code of all the intermediate words corresponding to the second ordered words is calculated as the propagation relationship embedding code between the second ordered word pairs, including: determining a first starting point relationship embedding code according to the propagation relationship embedding code between the second word and the fifth word; determining a first end point relationship embedding code according to the propagation relationship embedding code between the fifth word and the fourth word; determining a second starting point relationship embedding code according to the propagation relationship embedding code between the second word and the sixth word; determining a second end point relationship embedding code according to the propagation relationship embedding code between the sixth word and the fourth word; adding the product of the first starting point relationship embedding code and the first end point relationship embedding code to the product of the second starting point relationship embedding code and the second starting point relationship embedding code to generate the propagation relationship embedding code between the second ordered word pairs.

[0139] In some embodiments, summing up the preliminary relation embedding codes corresponding to each intermediate word to determine the propagation relation embedding code corresponding to the second ordered word pair may also include: summing up the preliminary relation embedding codes corresponding to each intermediate word to determine the intermediate relation embedding code corresponding to the second ordered word pair; determining the output ratio corresponding to the second ordered word pair based on the intermediate relation embedding code corresponding to the second ordered word pair, wherein the output ratio is used to determine the output ratio of the second ordered word pair when it is output; performing gate processing on the intermediate relation embedding code corresponding to the second ordered word pair according to the output ratio of the second ordered word pair when it is output, to obtain the propagation relation embedding code of the second ordered word pair.

[0140] The intermediate relation embedding code corresponding to the second ordered word pair can be processed by a linear transformation layer in the neural network to determine the output ratio corresponding to the second ordered word pair. The parameters of the linear transformation layer can be trained and adjusted.

[0141] Through the above method, the input gate and output gate can be calculated for each pair of word relationship representations during the relationship propagation process to control the relationship diffusion and absorption ratio.

[0142] Figure 7 The present invention is a flowchart of a method for determining a preliminary relation embedding code corresponding to an intermediate word according to an exemplary embodiment.

[0143] refer to Figure 7 , the method for determining the preliminary relation embedding code corresponding to the above-mentioned intermediate word may include the following steps.

[0144] Step S702, based on the propagation relationship embedding code of the first target word pair, determine the starting point relationship embedding code of the first target word pair, wherein the starting point relationship embedding code of the first target word pair is used to represent the corresponding features when the first target word pair serves as a starting point word pair (i.e., plays a role in the front).

[0145] Specifically, suppose that A and C can include two word pairs, AB and BC, and B and D can include two word pairs, BC and CD, where BC is in the second half of AC and in the first half of BD. Obviously, BC plays different roles in AC and BC. Then, it is necessary to calculate the corresponding features when BC is the starting word pair (i.e., it plays a role in the front) as the starting point relation embedding code, and it is necessary to calculate the corresponding features when BC is the end point word pair (i.e., it plays a role in the back) as the end point relation embedding code.

[0146] The propagation relation embedding code of the first target word pair can be processed by a linear transformation layer in the neural network to determine the starting relation embedding code of the first target word pair. The parameters of the linear transformation layer can be trained and adjusted.

[0147] In some embodiments, determining the starting point relationship embedding code of the first target word pair based on the propagation relationship embedding code of the first target word pair may include: determining the update ratio and the starting point relationship embedding code of the first target word pair based on the propagation relationship embedding code of the first target word pair; wherein the update ratio of the first target word pair is used to determine the contribution of the first target word pair in the parameter propagation process; updating the starting point relationship embedding code of the first target word pair according to the update ratio of the first target word pair, wherein the larger the update ratio of the first target word pair is, the more important the starting point relationship embedding code and the end point relationship embedding code corresponding to the first target word are.

[0148] The propagation relationship embedding coding of the first target word pair can be processed by a linear layer in the neural network to determine the update ratio of the first target word pair. The parameters of the linear layer can be trained and adjusted.

[0149] Through the above method, the input gate and output gate can be calculated for each pair of word relationship representations during the relationship propagation process to control the relationship diffusion and absorption ratio.

[0150] In some embodiments, the propagation relation embedding code of the first target word pair may be processed by a linear layer in a neural network to determine the starting relation embedding code of the first target word pair. The parameters of the linear layer may be trained and adjusted.

[0151] Step S704, determining the endpoint relationship embedding code of the second target word pair according to the propagation relationship embedding code of the second target word pair, wherein the endpoint relationship embedding code of the second target word is used to represent the corresponding features when the second target word pair is used as the endpoint word.

[0152] In some embodiments, the endpoint relationship embedding code of the second target word pair is determined based on the propagation relationship embedding code of the second target word pair, including: determining the update ratio and starting point relationship embedding code of the second target word pair based on the propagation relationship embedding code of the second target word pair; updating the starting point relationship embedding code of the second target word pair according to the update ratio of the second target word pair, wherein the larger the update ratio of the second target word pair is, the more important the starting point relationship embedding code and the endpoint relationship embedding code corresponding to the second target word are.

[0153] Step S706, multiply the starting point relation embedding code of the first target word pair and the ending point relation embedding code of the second target word pair to obtain a preliminary relation embedding code of the middle word.

[0154] The above method is based on AMR information fusion of relation propagation: the sparse AMR feature matrix is ​​densified through relation propagation, making its distribution closer to the self-attention matrix, thereby improving the effect of information fusion between the two.

[0155] Figure 8 It is a structural diagram corresponding to a text semantic analysis method shown according to an exemplary embodiment.

[0156] refer to Figure 8 As shown in the structural diagram, this application proposes a semantic analysis method, which specifically includes the following steps.

[0157] 1. Obtain the embedded code of the text to be analyzed through the pre-trained language model (PLM).

[0158] A pre-trained language model (PLM) is a model pre-trained on a large-scale text corpus. Its purpose is to learn common representations or features of language, which can be reused or fine-tuned in various natural language processing (NLP) applications.

[0159] 2. For the embedded code of the text to be analyzed, the self-attention weight is calculated based on the multi-head self-attention mechanism.

[0160] 3. Use the AMR (Abstract Meaning Representation, AMR) analysis tool to obtain the abstract semantic representation graph of the text to be analyzed. The nodes in the graph correspond to the words in the text one by one. (Align the graph nodes with the words in the text to be analyzed)

[0161] AMR graphs abstract words in sentences into concepts and relationships, and represent these concepts and relationships in the form of a single-rooted directed acyclic graph. This method can record the most important semantic information in natural language texts, but does not restrict the grammatical structure of actual expression.

[0162] 4. Through PLM, the node relationship names in the abstract semantic representation graph are mapped into vectors to obtain the graph relationship embedding code.

[0163] 5. Perform outer product operations on the word embedding codes of the text to be analyzed, and obtain the relational embedding code of the text to be analyzed.

[0164] 6. Add the relational embedding code of the abstract semantic representation graph to obtain the preliminary fused relational code.

[0165] 7. For the relation encoding after preliminary fusion, it is updated through the following relation propagation process, the purpose of which is to propagate the rare abstract semantic representation information to other word relation representations.

[0166] First, the corresponding update ratio can be calculated based on the relationship embedding coding between each pair of ordered words (which can be ordered words with known relationship embedding coding) through formulas (1) and (2): And output ratio

[0167]

[0168] in, Represents the relational embedding encoding of an ordered word pair consisting of the i-th word and the j-th word, where i and j are integers greater than 0.

[0169] LayerNorm normalizes each feature dimension of each sample, that is, subtracts the mean of each feature on the sample, and then divides it by the standard deviation of the feature on the sample. The purpose of this is to make the numerical distribution on each feature dimension more stable, avoiding the impact of changes in input distribution on network training.

[0170]

[0171] Then, it is required to calculate the corresponding starting point relation embedding code and end point relation embedding code based on the relational encoding of each pair of ordered words, and multiply them by the corresponding update ratio It can be understood that for each ordered word with a known relation embedding code, a set of corresponding starting point relation embedding codes and end point relation embedding codes can be obtained.

[0172]

[0173] Among them, ⊙ can represent the inner product calculation.

[0174] Among them, a ij Represents the starting point relation embedding encoding of ordered word pairs, b ij The endpoint relation embedding code representing the ordered word pair. The starting point relation embedding code and the endpoint relation embedding code of each word pair with known relation embedding code can be calculated by the above method.

[0175] For each pair of ordered words (which can be ordered words with unknown relationship embedding codes), calculate the sum of the bitwise multiplication of the starting embedding code and the end embedding code of all intermediate words, and use it as the update information of the pair of ordered words after passing through the linear layer.

[0176]

[0177] The above i and j can be integers greater than 0.

[0178] Where k can be a common neighbor of i and j. For example, n(i) can be defined as all neighbor nodes of node i, n(j) as all neighbor nodes of node j, and the value range of k is all intersections of n(i) and n(j). Two nodes with known relational embedding codes can be called neighbor nodes of each other.

[0179] Specifically, assuming that there are two intermediate words such as 2 and 3 between the ordered word pairs 1 and 4, the embedding relationship encoding between 1 and 4 can be calculated by the following formula: 12 ⊙b 24 +a 13 ⊙b 34 .

[0180] Finally, update the information of each pair of ordered words The corresponding output ratio Multiply them together to obtain the updated relational representation encoding of the pair of ordered words.

[0181]

[0182] 8. Add the updated relationship representation code to the attention weight calculated by the embedded code of the text to be analyzed in the second step as the new self-attention weight, take the embedded code of the text to be analyzed as the value of self-attention, and obtain the enhanced code of the text to be analyzed through the self-attention mechanism.

[0183] 9. The enhanced encoding of the text to be analyzed passes through the output layer to obtain the sentiment polarity classification of the words, and the evaluation object, evaluation content, and sentiment polarity triplet of the text to be analyzed are obtained through the decoding algorithm, which is the fine-grained sentiment analysis result.

[0184] The present application proposes a text encoding enhancement method, which introduces abstract semantic representation into the text encoding process to improve the encoding's ability to capture complete semantic information, thereby improving the accuracy of sentiment analysis tasks; designs a relationship propagation algorithm to densify the sparse abstract semantic representation, thereby more effectively integrating the abstract semantic representation with the text representation, generating enhanced text encoding, and ultimately enhancing the ability to capture key information (including sentiment elements, sentiment element relationships, and some key words), thereby improving the effect of fine-grained sentiment analysis tasks. The above embodiment, based on the introduction of abstract semantic representation to enhance text key information, proposes a relationship propagation algorithm that can propagate rare abstract semantic representation information in the full-text representation, and organically combine it with the full-text embedded coding, ultimately achieving an improvement in the effect of fine-grained sentiment analysis.

[0185] Compared with the existing distance-weighted attention, the text encoding enhancement method for sentiment analysis in this application significantly improves the model's key information capture capability and enhances the model's adaptability to long texts by introducing abstract semantic representation information and shortening the relative distance between sentiment elements. Compared with other methods of introducing abstract semantic information, this application designs a relationship propagation method to propagate abstract semantic representation information into the representation of the full text, further improving the model's performance on fine-grained sentiment analysis tasks.

[0186] It should be particularly pointed out that the various steps in the various embodiments of the above text analysis method can be mutually crossed, replaced, added, or deleted. Therefore, these reasonable permutations, combinations, and transformations of the text analysis method should also fall within the scope of protection of the present disclosure, and the scope of protection of the present disclosure should not be limited to the embodiments.

[0187] Based on the same inventive concept, the present disclosure also provides a text analysis device in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0188] Fig. 9 FIG. 1 is a block diagram of a text analysis device according to an exemplary embodiment. Fig. 9 The text analysis device 900 provided in the embodiment of the present disclosure may include: a text acquisition module 901, a self-attention weight determination module 902, an abstract semantic representation graph determination module 903, a graph relationship embedding coding determination module 904, an enhanced self-attention weight determination module 905 and an enhanced text coding determination module 906.

[0189] Among them, the text acquisition module 901 can be used to obtain the text to be analyzed and the embedded code corresponding to the text to be analyzed; the self-attention weight determination module 902 can be used to perform self-attention calculation on the embedded code corresponding to the text to be analyzed, and obtain the self-attention weight of each word in the text to be analyzed; the abstract semantic representation graph determination module 903 can be used to perform abstract semantic analysis on the text to be analyzed, and obtain the abstract semantic representation graph of the text to be analyzed, wherein the nodes in the abstract semantic representation graph correspond one-to-one to the words in the text to be analyzed; the graph relationship embedding code determination module 904 can be used to map the node relationship in the abstract semantic representation graph into a vector, and obtain the graph relationship embedding code; the enhanced self-attention weight determination module 905 can be used to enhance the self-attention weight of each word in the text to be analyzed through the graph relationship embedding code, and obtain the enhanced self-attention weight; the enhanced text code determination module 906 can be used to obtain the enhanced text code of the text to be analyzed based on the enhanced self-attention weight and the embedded code of each word in the text to be analyzed, so as to perform text analysis based on the enhanced text code.

[0190] It should be noted here that the above-mentioned text acquisition module 901, self-attention weight determination module 902, abstract semantic representation graph determination module 903, graph relationship embedding coding determination module 904, enhanced self-attention weight determination module 905 and enhanced text coding determination module 906 correspond to S202 to S212 in the method embodiment, and the examples and application scenarios implemented by the above-mentioned modules and corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned method embodiment. It should be noted that the above-mentioned modules as part of the device can be executed in a computer system such as a set of computer executable instructions.

[0191] In some embodiments, the enhanced self-attention weight determination module 905 may include: an outer product calculation submodule, a fusion submodule, and an enhancement submodule.

[0192] Among them, the outer product calculation submodule can be used to perform outer product calculation on words with association relationships in the text to be analyzed to obtain text relationship embedding code; the fusion submodule can be used to fuse the graph relationship embedding code and the text relationship embedding code to obtain the fused relationship code; the enhancement submodule can be used to enhance the self-attention weight of each word in the text to be analyzed through the fused relationship code to obtain the enhanced self-attention weight.

[0193] In some embodiments, the enhanced self-attention weight determination module 905 may include: a superposition submodule.

[0194] Among them, the superposition submodule can be used to superimpose the graph relationship embedding code on the self-attention weight of each word in the text to be analyzed to obtain the enhanced self-attention weight.

[0195] In some embodiments, the text to be analyzed includes a first word, a second word, and a third word; the graph relationship embedding code includes a first graph relationship embedding code for describing the relationship between the first word and the second word and a second graph relationship embedding code for describing the relationship between the first word and the third word; the self-attention weight corresponding to the first word includes a first self-attention weight for describing the influence of the second ordered word on the first word and a second self-attention weight for describing the influence of the third word on the first word; wherein the superposition submodule may include: a first superposition unit and a second superposition unit.

[0196] Among them, the first superposition unit can be used to superimpose the first graph relationship embedding code into the first self-attention weight to obtain the first enhanced self-attention weight, and the enhanced self-attention weight includes the first enhanced self-attention weight and the second enhanced self-attention weight; the second superposition unit can be used to superimpose the second graph relationship embedding code into the second self-attention weight to obtain the second enhanced self-attention weight.

[0197] In some embodiments, the self-attention weights of each word in the text to be analyzed are enhanced by graph relationship embedding coding to obtain the enhanced self-attention weights, including: performing relationship propagation according to the graph relationship embedding coding of the text to be analyzed, and determining the propagation relationship embedding coding between each pair of words in the text to be analyzed; enhancing the self-attention weights of each word in the text to be analyzed by propagation relationship embedding coding between each pair of words in the text to be analyzed, and obtaining the enhanced self-attention weights.

[0198] In some embodiments, relationship propagation is performed according to the graph relationship embedding code of the text to be analyzed to determine the propagation relationship embedding code between each pair of words in the text to be analyzed, including: performing relationship propagation according to the graph relationship embedding code of the text to be analyzed to determine the propagation relationship embedding code corresponding to the first ordered word pair in the text to be analyzed; wherein the first ordered word pair includes two words with known relationship embedding codes; determining the second ordered word pair in the text to be analyzed, wherein the relationship embedding code between the second ordered word pair is unknown; determining the first target word pair and the second target word pair according to the middle word between the second ordered word pair, wherein the first target word pair is composed of the first word and the middle word of the second ordered word pair, and the second target word pair is composed of the middle word and the last word of the second ordered word pair; the first ordered word pair includes the first target word pair and the second target word pair; determining the preliminary relationship embedding code corresponding to the middle word according to the relationship embedding code of the first target word pair and the second target word pair; summing the preliminary relationship embedding codes corresponding to the middle words to determine the propagation relationship embedding code corresponding to the second ordered word pair.

[0199] In some embodiments, a preliminary relation embedding code corresponding to an intermediate word is determined based on the relation embedding code of a first target word pair and a second target word pair, including: determining a starting point relation embedding code of the first target word pair based on the propagation relation embedding code of the first target word pair, wherein the starting point relation embedding code of the first target word pair is used to represent the corresponding features when the first target word pair is used as the starting point word pair; determining an end point relation embedding code of the second target word pair based on the propagation relation embedding code of the second target word pair, wherein the end point relation embedding code of the second target word is used to represent the corresponding features when the second target word pair is used as the end point word; multiplying the starting point relation embedding code of the first target word pair and the end point relation embedding code of the second target word pair to obtain a preliminary relation embedding code of the intermediate word.

[0200] In some embodiments, based on the propagation relationship embedding code of the first target word pair, the starting point relationship embedding code of the first target word pair is determined, including: determining the update ratio and the starting point relationship embedding code of the first target word pair based on the propagation relationship embedding code of the first target word pair; wherein the update ratio of the first target word pair is used to determine the contribution of the first target word pair in the parameter propagation process; updating the starting point relationship embedding code of the first target word pair by the update ratio of the first target word pair, wherein the larger the update ratio of the first target word pair, the more important the starting point relationship embedding code and the end point relationship embedding code corresponding to the first target word; wherein, based on the propagation relationship embedding code of the second target word pair, determining the end point relationship embedding code of the second target word pair, including: determining the update ratio and the starting point relationship embedding code of the second target word pair based on the propagation relationship embedding code of the second target word pair; updating the starting point relationship embedding code of the second target word pair by the update ratio of the second target word pair, wherein the larger the update ratio of the second target word pair, the more important the starting point relationship embedding code and the end point relationship embedding code corresponding to the second target word.

[0201] In some embodiments, the preliminary relation embedding codes corresponding to the intermediate words are summed to determine the propagation relation embedding code corresponding to the second ordered word pair, including: summing the preliminary relation embedding codes corresponding to the intermediate words to determine the intermediate relation embedding code corresponding to the second ordered word pair; determining the output ratio corresponding to the second ordered word pair according to the intermediate relation embedding code corresponding to the second ordered word pair, the output ratio being used to determine the output ratio of the second ordered word pair when it is output; performing gate processing on the intermediate relation embedding code corresponding to the second ordered word pair according to the output ratio of the second ordered word pair when it is output, to obtain the propagation relation embedding code of the second ordered word pair.

[0202] Since the functions of the device 900 have been described in detail in the corresponding method embodiments, the present disclosure will not elaborate on them here.

[0203] The modules and / or submodules and / or units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The modules and / or submodules and / or units described may also be arranged in a processor. The names of these modules and / or submodules and / or units do not, in certain circumstances, constitute limitations on the modules and / or submodules and / or units themselves.

[0204] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a part of a module or program segment, and a part of the above-mentioned module or program segment contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer program instructions.

[0205] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0206] Fig.10 FIG. 1 shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure. It should be noted that: Fig.10 The electronic device 1000 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0207] like Fig.10 As shown, the electronic device 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0208] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0209] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains computer program instructions for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1009, and / or installed from a removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, the above-mentioned functions defined in the system of the present disclosure are executed.

[0210] It should be noted that the computer-readable storage medium shown in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable computer program instructions. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The computer program instructions contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0211] As another aspect, the present disclosure also provides a computer-readable storage medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by a device, the device can realize functions including: obtaining the text to be analyzed and the embedded code corresponding to the text to be analyzed; performing self-attention calculation on the embedded code corresponding to the text to be analyzed, and obtaining the self-attention weight of each word in the text to be analyzed; performing abstract semantic analysis on the text to be analyzed, and obtaining an abstract semantic representation graph of the text to be analyzed, wherein the nodes in the abstract semantic representation graph correspond to the words in the text to be analyzed one by one; mapping the node relationship in the abstract semantic representation graph to a vector, and obtaining a graph relationship embedding code; enhancing the self-attention weight of each word in the text to be analyzed through the graph relationship embedding code, and obtaining the enhanced self-attention weight; based on the enhanced self-attention weight and the embedded code of each word in the text to be analyzed, obtaining the enhanced text code of the text to be analyzed, so as to perform text analysis based on the enhanced text code.

[0212] According to one aspect of the present disclosure, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer program instructions, the computer program instructions being stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium, and a processor executes the computer program instructions to implement the methods provided in various optional implementations of the above embodiments.

[0213] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), including a number of computer program instructions for enabling an electronic device (which can be a server or a terminal device, etc.) to execute the method according to the embodiment of the present disclosure.

[0214] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and embodiments are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0215] It should be understood that the present disclosure is not limited to the detailed structures, drawings or implementations shown herein, but rather the present disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A text analysis method, characterized in that: include: Obtaining the text to be analyzed and the embedded code corresponding to the text to be analyzed; Performing self-attention calculation on the embedded code corresponding to the text to be analyzed to obtain the self-attention weight of each word in the text to be analyzed; Performing abstract semantic analysis on the text to be analyzed to obtain an abstract semantic representation graph of the text to be analyzed, wherein nodes in the abstract semantic representation graph correspond one-to-one to words in the text to be analyzed; Mapping the node relationships in the abstract semantic representation graph into vectors to obtain graph relationship embedding codes; The self-attention weight of each word in the text to be analyzed is enhanced by using the graph relationship embedding coding to obtain the enhanced self-attention weight; Based on the enhanced self-attention weights and the embedded codes of each word in the text to be analyzed, an enhanced text code of the text to be analyzed is obtained, so as to perform text analysis based on the enhanced text code.

2. The method according to claim 1, characterized in that: The self-attention weight of each word in the text to be analyzed is enhanced by the graph relationship embedding coding to obtain the enhanced self-attention weight, including: Perform outer product calculation on the words with association relationship in the text to be analyzed to obtain text relationship embedding code; The graph relationship embedding code and the text relationship embedding code are merged to obtain; The self-attention weight of each word in the text to be analyzed is enhanced through the fused relational encoding to obtain the enhanced self-attention weight.

3. The method according to claim 1, characterized in that: The self-attention weight of each word in the text to be analyzed is enhanced by the graph relationship embedding coding to obtain the enhanced self-attention weight, including: Perform relation propagation according to the graph relation embedding code of the text to be analyzed, and determine the propagation relation embedding code between two words in the text to be analyzed; The self-attention weight of each word in the text to be analyzed is enhanced by embedding the propagation relationship between each two words in the text to be analyzed to obtain the enhanced self-attention weight.

4. The method according to claim 3, characterized in that: Performing relation propagation according to the graph relation embedding code of the text to be analyzed, and determining the propagation relation embedding code between two words in the text to be analyzed, includes: Perform relation propagation according to the graph relation embedding code of the text to be analyzed, and determine the propagation relation embedding code corresponding to the first ordered word pair in the text to be analyzed; wherein the first ordered word pair includes two words with known relation embedding codes; Determining a second ordered word pair in the text to be analyzed, wherein the relationship embedding code between the second ordered word pairs is unknown; Determine a first target word pair and a second target word pair according to the middle word between the second ordered word pairs, wherein the first target word pair is composed of the first word of the second ordered word pair and the middle word, and the second target word pair is composed of the middle word and the last word of the second ordered word pair; and the first ordered word pair includes the first target word pair and the second target word pair; Determine a preliminary relational embedding code corresponding to the intermediate word according to the relational embedding code of the first target word pair and the second target word pair; The preliminary relation embedding codes corresponding to the intermediate words are summed to determine the propagation relation embedding codes corresponding to the second ordered word pairs.

5. The method according to claim 4, characterized in that: Determining a preliminary relational embedding code corresponding to the intermediate word according to the relational embedding code of the first target word pair and the second target word pair includes: Determine a starting point relation embedding code of the first target word pair according to the propagation relation embedding code of the first target word pair, wherein the starting point relation embedding code of the first target word pair is used to represent a feature corresponding to when the first target word pair is used as a starting point word pair; Determine the endpoint relation embedding code of the second target word pair according to the propagation relation embedding code of the second target word pair, wherein the endpoint relation embedding code of the second target word is used to represent the corresponding features when the second target word pair is used as an endpoint word; The starting point relation embedding code of the first target word pair and the ending point relation embedding code of the second target word pair are multiplied to obtain a preliminary relation embedding code of the middle word.

6. The method according to claim 4, characterized in that: The preliminary relation embedding codes corresponding to the intermediate words are summed to determine the propagation relation embedding codes corresponding to the second ordered word pairs, including: Sum the preliminary relation embedding codes corresponding to the intermediate words to determine the intermediate relation embedding codes corresponding to the second ordered word pairs; Determine the output ratio corresponding to the second ordered word pair according to the intermediate relation embedding code corresponding to the second ordered word pair, wherein the output ratio is used to determine the output ratio when the second ordered word pair is output; The intermediate relation embedding code corresponding to the second ordered word pair is gated according to the output ratio of the second ordered word pair at the time of output, so as to obtain the propagation relation embedding code of the second ordered word pair.

7. A text analysis device, characterized in that: include: A text acquisition module, used to acquire the text to be analyzed and the embedded code corresponding to the text to be analyzed; A self-attention weight determination module is used to perform self-attention calculation on the embedded code corresponding to the text to be analyzed, and obtain the self-attention weight of each word in the text to be analyzed; An abstract semantic representation graph determination module is used to perform abstract semantic analysis on the text to be analyzed to obtain an abstract semantic representation graph of the text to be analyzed, wherein the nodes in the abstract semantic representation graph correspond one-to-one to the words in the text to be analyzed; A graph relationship embedding code determination module, used to map the node relationship in the abstract semantic representation graph into a vector to obtain a graph relationship embedding code; An enhanced self-attention weight determination module is used to enhance the self-attention weight of each word in the text to be analyzed through the graph relationship embedding coding to obtain the enhanced self-attention weight; The enhanced text encoding determination module is used to obtain the enhanced text encoding of the text to be analyzed based on the enhanced self-attention weight and the embedded encoding of each word in the text to be analyzed, so as to perform text analysis based on the enhanced text encoding.

8. An electronic device, characterized in that: include: Memory and processor; The memory is used to store computer program instructions; the processor calls the computer program instructions stored in the memory to implement the text analysis method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the text analysis method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising computer program instructions, wherein the computer program instructions are stored in a computer-readable storage medium, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images