Method, apparatus, and storage medium for generating cross-paragraph text semantic representation vectors

Abstract

An embodiment of the present application provides a method, an apparatus, and a storage medium for generating a cross-paragraph text semantic representation vector, including: obtaining a first semantic representation vector and a multi-granularity graph of the cross-paragraph text according to the cross-paragraph text; obtaining a graph node feature vector of the cross-paragraph text according to the first semantic representation vector of the cross-paragraph text and the multi-granularity graph; obtaining a second semantic representation vector of the cross-paragraph text, where both the first semantic representation vector and the second semantic representation vector indicate the semantic information of the cross-paragraph text, and the second semantic representation vector is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector. Since the present solution involves more types of granularity, the cross-paragraph text semantic representation vector of the present solution contains richer and more comprehensive semantic information, which helps to improve the accuracy of sorting candidate paths for subsequent searches and also helps to improve the accuracy of extracting question-and-answer answers.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and particularly relates to a method, apparatus, and storage medium for generating cross-paragraph text semantic representation vectors. Background Art

[0002] Currently, deep pre-trained language models are the mainstream technology in natural language processing tasks such as search and question answering. Deep pre-trained language models can generate semantic representation vectors for input text. However, when the core structure - the fully connected attention layer extracts semantic information, it completely depends on the model's learning of the relationships between identifier tokens. Tokens are obtained based on text segmentation, so it completely discards the additional prior information that may be obtained, as Figure 1a shown. For complex scenarios such as search and question answering, the input involves multiple paragraphs. Using deep pre-trained language models has problems such as more input text, more complex objects and relationships between objects in the text, and the need to extract cross-paragraph semantic information. This makes the performance of the method that completely depends on the model to extract semantic information by learning the relationships between tokens decline severely, and the difficulty of generating text semantic representation vectors by deep pre-trained language models increases significantly.

[0003] In the prior art, for complex scenarios such as search and question answering, in order to better generate cross-paragraph text semantic representation vectors based on deep pre-trained language models, based on the semantic connections between entities, information interaction is carried out at the entity granularity to generate graph node feature vectors in the entity graph, and the graph node feature vectors are fused with the text semantic representation vectors generated by the deep pre-trained language models to obtain the final cross-paragraph text semantic representation vectors. The cross-paragraph text semantic representation vectors generated by this method incorporate, in addition to the semantic information extracted by the deep pre-trained language models, the additional prior information of the semantic connections between entities, as Figure 1b shown.

[0004] However, due to the overly single semantic connections at the entity granularity, this technology cannot effectively solve the problem of weak semantic connections between multiple web page paragraphs in the input of complex search and question answering task scenarios, nor can it focus on key semantic information for longer input text, more complex objects and relationships between objects, and the improvement effect on cross-paragraph text semantic representation vectors is not obvious. Summary of the Invention

[0005] This application discloses a method, apparatus, and storage medium for generating cross-paragraph text semantic representation vectors, which can generate cross-paragraph text semantic representation vectors with richer and more comprehensive semantics.

[0006] First aspect, an embodiment of the present application provides a method for generating a semantic representation vector of cross-paragraph text, including: obtaining a first semantic representation vector and a multi-granularity graph of the cross-paragraph text according to the cross-paragraph text, where the cross-paragraph text includes at least two paragraphs, and the multi-granularity graph indicates the relationships between sub-texts of multiple granularities within the same paragraph in the cross-paragraph text, as well as the relationships between sub-texts of multiple granularities across paragraphs; obtaining a graph node feature vector of the cross-paragraph text according to the first semantic representation vector of the cross-paragraph text and the multi-granularity graph, where the graph node feature vector indicates the semantic information of sub-texts of multiple granularities within the same paragraph and across paragraphs in the cross-paragraph text; obtaining a second semantic representation vector of the cross-paragraph text, where both the first semantic representation vector and the second semantic representation vector indicate the semantic information of the cross-paragraph text, and the second semantic representation vector is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector. The above second semantic representation vector is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector, that is, by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector. This fusion is based on the first semantic representation vector of the cross-paragraph text, and fuses the relationships between sub-texts of multiple granularities within the same paragraph and across paragraphs in the cross-paragraph text to obtain the above second semantic representation vector. Fusion means obtaining one feature vector based on two feature vectors, and this solution does not specifically limit how to achieve the fusion. For example, fusion can be performed by means of vector superposition, or other means can also be used for fusion.

[0007] Through the embodiment of the present application, by obtaining the first semantic representation vector and the multi-granularity graph of the cross-paragraph text, the graph node feature vector of the cross-paragraph text is further obtained, and the second semantic representation vector of the cross-paragraph text is obtained according to the above first semantic representation vector and the graph node feature vector. Compared with the semantic representation vector of the cross-paragraph text obtained only based on the single granularity of entity granularity in the prior art, this solution is based on the relationships between multi-granularity texts within the same paragraph and across paragraphs in the cross-paragraph text, and obtains the semantic representation vector of the cross-paragraph text based on multiple granularities. Since the types of granularities involved in this solution are more, the semantic information included in generating the semantic representation vector of the cross-paragraph text is richer and more comprehensive, which helps to improve the accuracy of sorting candidate paths for subsequent search, and also helps to improve the accuracy of extracting question-and-answer answers.

[0008] As an optional implementation manner, the relationships between the sub-texts of multiple granularities include the relationships between sub-texts of the same granularity and the relationships between sub-texts of different granularities.

[0009] Based on various relationships captured in the cross-paragraph text, the purpose of fully expressing the semantic connections of the cross-paragraph text at different levels is achieved.

[0010] As an alternative implementation, the multi-granularity graph includes nodes of at least two granularities and edges for connecting two nodes. Among them, each type of node with a specific granularity in the at least two types of granularity nodes indicates a sub-text of a certain granularity, and the edge indicates the relationship between the two connected nodes, and the relationship is either the relationship between sub-texts of the same granularity or the relationship between sub-texts of different granularities.

[0011] Based on the text organizational structure of the cross-paragraph text and various relationships captured in the cross-paragraph text, multi-granularity graph construction is performed, achieving the purpose of fully expressing the semantic connections of the cross-paragraph text at different levels.

[0012] As an alternative implementation, a first intermediate result is obtained according to the first semantic representation vector of the cross-paragraph text and the sub-texts carried by nodes of multiple granularities in the multi-granularity graph; a second intermediate result is obtained according to the first intermediate result and the relationship between sub-texts of the same granularity; and a graph node feature vector of the cross-paragraph text is obtained according to the second intermediate result and the relationship between sub-texts of different granularities. The relationship between sub-texts of the same granularity involves multiple granularities.

[0013] In this solution, not only information interaction within nodes of the same granularity is performed on the graph node feature vector, enabling the graph node feature vector to fully capture the cross-paragraph semantic connections at different levels of semantic information; but also information interaction between nodes of different granularities is performed on the graph node feature vector, making the feature vector of each type of node contain different levels of semantic information from macroscopic to local, which makes the information contained in the graph node feature vector more comprehensive and rich.

[0014] As an alternative implementation, the first intermediate result and a first sub-graph are input into a first neural network to obtain the first intermediate result after the first sub-graph is updated. The first sub-graph includes nodes of the first granularity in the multi-granularity graph and the edges corresponding to the nodes of the first granularity; the first intermediate result and a second sub-graph are input into a second neural network to obtain the first intermediate result after the second sub-graph is updated. The second sub-graph includes nodes of the second granularity in the multi-granularity graph and the edges corresponding to the nodes of the second granularity; and the second intermediate result is obtained according to the first intermediate result after the first sub-graph is updated and the first intermediate result after the second sub-graph is updated.

[0015] Among them, the multi-granularity graph includes at least two of paragraph granularity, sentence granularity, and entity granularity.

[0016] Among them, starting from the current first-line indentation of the paragraph and ending with the first-line indentation of the adjacent paragraph, paragraph division can be performed, and then nodes at the paragraph granularity can be obtained. Sentence division is performed based on punctuation marks such as commas, semicolons, full stops, exclamation marks, ellipsis, etc., and then nodes at the sentence granularity can be obtained. The entity in this solution refers to an instance of a certain concept, and the entity is text or a label. For example, entities are mainly words or phrases with clear referents such as nouns, numerals, etc., such as personal names, place names, organization names, proper nouns, etc., as well as words representing concepts such as time, quantity, currency, proportional values, etc. Entity recognition can be performed through entity recognition technology, and then nodes at the entity granularity can be obtained.

[0017] As an optional implementation manner, the relationships include at least one of the following: inclusion, inclusion of the same entity, entity co-reference, and web hyperlink.

[0018] Among them, the above inclusion relationship includes relationships such as a paragraph including sentences, and a sentence including entities. The relationship of including the same entity indicates sentences with the same entity. The entity co-reference relationship indicates entities with the same meaning. The web hyperlink relationship indicates a web hyperlink relationship between at least two paragraphs.

[0019] As an optional implementation manner, the first neural network and the second neural network are different heterogeneous graph neural networks.

[0020] In a second aspect, an apparatus for generating a cross-paragraph text semantic representation vector according to an embodiment of the present application includes: a first generation module, configured to obtain a first semantic representation vector and a multi-granularity graph of the cross-paragraph text according to the cross-paragraph text, where the cross-paragraph text includes at least two paragraphs, and the multi-granularity graph indicates relationships between sub-texts of multiple granularities within the same paragraph in the cross-paragraph text, and relationships between sub-texts of multiple granularities across paragraphs; a second generation module, configured to obtain a graph node feature vector of the cross-paragraph text according to the first semantic representation vector of the cross-paragraph text and the multi-granularity graph, where the graph node feature vector indicates semantic information of sub-texts of multiple granularities within the same paragraph and across paragraphs in the cross-paragraph text; a third generation module, configured to obtain a second semantic representation vector of the cross-paragraph text, where both the first semantic representation vector and the second semantic representation vector indicate semantic information of the cross-paragraph text, and the second semantic representation vector is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector.

[0021] Optionally, the multi-granularity graph includes nodes of at least two granularities and edges for connecting two nodes. Among them, each granularity of nodes in the at least two granularities of nodes indicates a sub-text of a granularity, and the edge indicates the relationship between the two connected nodes, and the relationship is a relationship between sub-texts of the same granularity or a relationship between sub-texts of different granularities.

[0022] Among them, the second generation module is used to: obtain a first intermediate result according to the first semantic representation vector of the cross-paragraph text and the sub-text carried by nodes of multiple granularities in the multi-granularity graph; obtain a second intermediate result according to the first intermediate result and the relationship between sub-texts of the same granularity; and obtain the graph node feature vector of the cross-paragraph text according to the second intermediate result and the relationship between sub-texts of different granularities.

[0023] Optionally, the multi-granularity graph includes a first granularity and a second granularity. The second generation module is used to: input the first intermediate result and the first sub-graph into a first neural network to obtain the updated first intermediate result of the first sub-graph. The first sub-graph includes nodes of the first granularity in the multi-granularity graph and the edges corresponding to the nodes of the first granularity; input the first intermediate result and the second sub-graph into a second neural network to obtain the updated first intermediate result of the second sub-graph. The second sub-graph includes nodes of the second granularity in the multi-granularity graph and the edges corresponding to the nodes of the second granularity; and obtain the second intermediate result according to the updated first intermediate result of the first sub-graph and the updated first intermediate result of the second sub-graph.

[0024] Optionally, the multi-granularity graph includes at least two of paragraph granularity, sentence granularity, and entity granularity.

[0025] Optionally, the relationship includes at least one of the following: inclusion, containing the same entity, entity co-reference, and web hyperlink.

[0026] Optionally, the first neural network and the second neural network are different heterogeneous graph neural networks.

[0027] In a third aspect, an embodiment of the present application provides a device for generating a semantic representation vector of a cross-paragraph text, including a processor and a memory; wherein, the memory is used to store program code, and the processor is used to call the program code to execute the method described above.

[0028] In a fourth aspect, the present application provides a computer storage medium, including computer instructions. When the computer instructions run on an electronic device, the electronic device is enabled to execute the method provided by any possible implementation manner of the first aspect.

[0029] Fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on a computer, it causes the computer to execute the method provided by any possible implementation manner of the first aspect.

[0030] It can be understood that the devices described in the second aspect, the devices described in the third aspect, the computer storage medium described in the fourth aspect, or the computer program product described in the fifth aspect provided above are all used to execute the method provided in any one of the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The drawings used in the embodiments of the present application will be introduced below.

[0032] Figure 1a is a schematic diagram of a fully connected attention layer of a deep pre-trained language model in the prior art;

[0033] Figure 1b is a schematic flowchart of a method for generating a cross-paragraph text semantic representation vector in the prior art;

[0034] Figure 2a is a natural language processing system provided by an embodiment of the present application;

[0035] Figure 2b is another natural language processing system provided by an embodiment of the present application;

[0036] Figure 2c is another natural language processing system provided by an embodiment of the present application;

[0037] Figure 2d is a schematic diagram of a system architecture provided by an embodiment of the present application;

[0038] Figure 3 is a schematic flowchart of a method for generating a cross-paragraph text semantic representation vector provided by an embodiment of the present application;

[0039] Figure 4a is a schematic diagram of node division provided by an embodiment of the present application;

[0040] Figure 4b is a schematic diagram of the edges of nodes within a paragraph provided by an embodiment of the present application;

[0041] Figure 4c is a schematic diagram of a multi-granularity graph provided by an embodiment of the present application;

[0042] Figure 4d is a schematic diagram of a subgraph of paragraph-level nodes provided by an embodiment of the present application;

[0043] Figure 4e It is a schematic diagram of a sub - graph of sentence - level nodes provided by an embodiment of the present application;

[0044] Figure 4f It is a schematic diagram of a sub - graph of nodes with different granularities provided by an embodiment of the present application;

[0045] Figure 5 It is a schematic flow chart of a method for generating cross - paragraph text semantic representation vectors provided by an embodiment of the present application;

[0046] Figure 6a It is a schematic diagram of a multi - granularity graph provided by an embodiment of the present application;

[0047] Figure 6b It is a schematic diagram of updating the feature vector of a graph node provided by an embodiment of the present application;

[0048] Figure 6c It is another schematic diagram of updating the feature vector of a graph node provided by an embodiment of the present application;

[0049] Figure 7 It is a schematic flow chart of a method for generating cross - paragraph text semantic representation vectors provided by an embodiment of the present application;

[0050] Figure 8 It is a schematic structural diagram of a device for generating cross - paragraph text semantic representation vectors provided by an embodiment of the present application;

[0051] Figure 9 It is another schematic structural diagram of a device for generating cross - paragraph text semantic representation vectors provided by an embodiment of the present application. Detailed implementation manners

[0052] The embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. The terms used in the implementation part of the embodiments of the present application are only for explaining the specific embodiments of the present application, and are not intended to limit the present application.

[0053] Scenario introduction

[0054] As Figure 2a shown, a natural language processing system includes a user device and a data processing device.

[0055] The user device includes a user and intelligent terminals such as a mobile phone, a personal computer, or an information processing center. The user device is the initiating end of natural language data processing and is the initiator of requests such as language answering or querying. Usually, the user initiates requests through the user device.

[0056] The data processing device may be a device or server with data processing functions such as a cloud server, a network server, an application server, and a management server. The data processing device receives query statements / voices / texts and other question sentences from the intelligent terminal through the interaction interface, and then performs language data processing in ways such as machine learning, deep learning, searching, reasoning, and decision-making through the memory for storing data and the processor for data processing. The memory may be a general term, including local storage and a database for storing historical data. The database may be on the data processing device or on other network servers.

[0057] As Figure 2b shown, it is another application scenario of the natural language processing system. In this scenario, the intelligent terminal directly serves as the data processing device, directly receives the input from the user, and is directly processed by the hardware of the intelligent terminal itself. The specific process is similar to Figure 2a and can refer to the above description, which will not be elaborated here.

[0058] As Figure 2c shown, the user device may be the local device 101 or 102, and the data processing device may be the execution device 110. The data storage system 150 may be integrated on the execution device 110 or may be set on the cloud or other network servers.

[0059] See Appendix Figure 2d . The embodiment of the present invention provides a system architecture 200. The data acquisition device 260 is used to acquire training data and store it in the database 230. The training device 220 generates a cross-paragraph text semantic representation vector model 201 based on the training data maintained in the database 230. How the training device 220 obtains the cross-paragraph text semantic representation vector model 201 based on the training data will be described in more detail below. The generated cross-paragraph text semantic representation vector model can determine the web page sorting of the candidate path or the final answer to the question according to the question sentence.

[0060] Figure 2d shown is the functional module diagram in the data processing process. Corresponding to Figures 2a - 2c the actual application scenario diagram in Figures 2a - 2c , the user device 240 may be the user device in Figures 2a - 2c . When the data processing capabilities of the user device in Figure 2a are relatively strong, the execution device 210 and the data storage system 250 may be integrated in the user device. In some embodiments, the execution device 210 and the data storage system 250 may also be integrated on the data processing device in Figure 2aOn the data processing device in it, it can be set on other servers on the cloud or network.

[0061] In the field of NLP, the data acquisition device 260 can be a terminal device, or an input / output interface of a server or the cloud, and is an interaction layer (interface) for obtaining query statements and returning reply statements.

[0062] The above cross-paragraph text semantic representation vector model 201 can be composed of a deep neural network. Among them, the work of each layer in the deep neural network can be described by a mathematical expression as follows: From a physical level, the work of each layer in the deep neural network can be understood as completing the transformation from the input space (the set of input vectors) to the output space (i.e., from the row space to the column space of the matrix) through five operations on the input space. These five operations include: 1. Dimension increase / dimension decrease; 2. Magnification / shrinkage; 3. Rotation; 4. Translation; 5. "Bending". Among them, the operations of 1, 2, and 3 are completed by ; the operation of 4 is completed by +b, and the operation of 5 is implemented by α(). The reason for using the word "space" here is that the object to be classified is not a single thing, but a class of things. Space refers to the set of all individuals of this class of things. Among them, W is the weight vector, and each value in this vector represents the weight value of a neuron in this layer of the neural network. This vector W determines the space transformation from the input space to the output space described above, that is, the weight W of each layer controls how to transform the space. The purpose of training a deep neural network is to finally obtain the weight matrices of all layers of the trained neural network (the weight matrix formed by vectors W of many layers). Therefore, the training process of the neural network is essentially a process of learning the way to control space transformation, and more specifically, learning the weight matrix.

[0063] Because it is hoped that the output of the deep neural network is as close as possible to the value that is really wanted to be predicted, the weight vector of each layer of the neural network can be updated by comparing the predicted value of the current network with the really wanted target value and then according to the difference between the two (of course, there is usually an initialization process before the first update, that is, pre-configuring parameters for each layer in the deep neural network). For example, if the predicted value of the network is high, adjust the weight vector to make it predict lower, and keep adjusting until the neural network can predict the really wanted target value. Therefore, it is necessary to pre-define "how to compare the difference between the predicted value and the target value", which is the loss function or objective function. They are important equations for measuring the difference between the predicted value and the target value. Among them, taking the loss function as an example, the higher the output value (loss) of the loss function, the greater the difference. Then the training of the deep neural network becomes a process of minimizing this loss as much as possible.

[0064] The cross-paragraph text semantic representation vector model 201 obtained by the training device 220 can be applied to different systems or devices. In Figure 2d , the execution device 210 is configured with an I / O interface 212 to interact with external devices, and the "user" can input data to the I / O interface 212 through the user device 240.

[0065] The execution device 210 can call data, code, etc. in the data storage system 250, and can also store data, instructions, etc. in the data storage system 250.

[0066] The associated function module 213 preprocesses the received question statement to facilitate the subsequent generation process of the cross-paragraph text semantic representation vector for the question statement. Optionally, the associated function module 213 can also post-process the generated cross-paragraph text semantic representation vector to output statements / voices / text, etc.

[0067] Finally, the I / O interface 212 returns the processing result to the user device 240 and provides it to the user.

[0068] More deeply, the training device 220 can generate corresponding cross-paragraph text semantic representation vector models 201 based on different data for different targets to provide better results for users.

[0069] In Figure 2d In the shown case, the user can manually specify the data input to the execution device 210. For example, operate in the interface provided by the I / O interface 212. In another case, the user device 240 can automatically input data to the I / O interface 212 and obtain the result. If the user device 240 needs to obtain the user's authorization for automatic data input, the user can set the corresponding permissions in the user device 240. The user can view the result output by the execution device 210 in the user device 240, and the specific presentation form can be specific ways such as display, sound, action, etc. The user device 240 can also be used as a data acquisition end to store the collected training data in the database 230.

[0070] It should be noted that Figure 2d is only a schematic diagram of a system architecture provided by an embodiment of the present invention. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 2d , the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 can also be placed in the execution device 210.

[0071] The following provides a detailed introduction to the method for generating cross-paragraph text semantic representation vectors provided by the embodiments of the present application. Among them, the embodiments of the present application are applicable to complex search scenarios and complex question-and-answer scenarios. A complex search scenario means that the query input by the user is related to the information of at least two web pages. Specifically, the search engine needs to output multiple paths in descending order of the relevance between the query and the paths, and each path includes at least two web pages. This path can be understood as a set composed of two or more web page paragraphs with semantic connections. A complex question-and-answer scenario means that the question input by the user needs to obtain the correct answer based on the information of at least two web pages. Specifically, the question-and-answer system needs to extract the answer according to an optimal path recalled, and the optimal path is the path from which the correct answer is most likely to be extracted. That is to say, the embodiments of the present application are applicable to scenarios where the search / question-and-answer results are determined based on at least two web pages.

[0072] Refer to Figure 3 As shown, it is a schematic flowchart of a method for generating cross-paragraph text semantic representation vectors provided by the embodiments of the present application. This method specifically includes steps 301-303, which are as follows:

[0073] 301. Obtain the first semantic representation vector and the multi-granularity graph of the cross-paragraph text according to the cross-paragraph text, where the cross-paragraph text includes at least two paragraphs, and the multi-granularity graph indicates the relationships between sub-texts of multiple granularities within the same paragraph in the cross-paragraph text, as well as the relationships between sub-texts of multiple granularities across paragraphs;

[0074] Among them, before step 301, the method may further include steps 3001-3003, which are as follows:

[0075] 3001. Receive the Query / Question input by the user;

[0076] This Query represents the text content input by the user in the search box, that is, the "query" input by the user. For example, the user conducts relevant information queries in the search engine.

[0077] This Question represents the question input by the user. For example, the user conducts question-and-answer operations based on question-and-answer robots, smart speakers, etc.

[0078] 3002. Obtain M candidate paths according to the Query / Question input by the user, where each candidate path includes at least two web pages;

[0079] This candidate path represents the source of the current web page as a candidate query result or the answer to the question. Among them, based on the above M candidate paths, by sorting the M candidate paths, the final query result / question answer is obtained.

[0080] 3003. Obtain M cross-paragraph texts based on the Query / Question and at least two web pages of each of the M candidate paths.

[0081] Optionally, obtain the first paragraph of each web page among at least two web pages of each candidate path, and then splice the Query / Question and the first paragraph of each web page among at least two web pages of each candidate path to obtain M cross-paragraph texts.

[0082] That is to say, each cross-paragraph text includes at least two paragraphs. In the above embodiment, splicing is performed taking the first paragraph of each web page as an example. As another implementation, splicing can be performed taking the last paragraph of each web page as an example, etc. Of course, it can also be splicing of any other paragraphs, such as splicing the first paragraph of the first web page and the second paragraph of the second web page, etc. This solution does not make specific limitations on this.

[0083] Specifically, the first semantic representation vector of each of the above cross-paragraph texts can be obtained by inputting the cross-paragraph text into a deep pre-trained language model.

[0084] It should be noted that in the embodiments of the present application, M is taken as 1 for illustration.

[0085] The multi-granularity graph includes nodes of at least two granularities and edges for connecting two nodes. Among them, each node of each of the at least two granularities of nodes indicates a sub-text of a granularity, the edge indicates the relationship between the two connected nodes, and the relationship is the relationship between sub-texts of the same granularity or the relationship between sub-texts of different granularities.

[0086] As an implementation, obtaining the multi-granularity graph of the cross-paragraph text according to the above includes steps 301A-301C, specifically as follows:

[0087] 301A. Obtain nodes corresponding to each of the at least two paragraphs according to the at least two paragraphs in the cross-paragraph text, where each node corresponding to each paragraph includes nodes of at least two granularities;

[0088] For any one of the above cross-paragraph texts, adopt techniques such as sentence splitting technology and entity recognition technology to obtain nodes of different granularities corresponding to each of the at least two paragraphs.

[0089] Specifically, starting from the current first-line indentation and ending with the adjacent first-line indentation, paragraph division can be performed to obtain nodes at the paragraph granularity. Sentence division is performed based on punctuation marks such as commas, semicolons, full stops, exclamation marks, ellipsis, etc., to obtain nodes at the sentence granularity. The entities in this solution refer to instances of a certain concept, mainly including nouns such as personal names, place names, organization names, proper nouns, etc., as well as words representing concepts such as time, quantity, currency, proportional values, etc. Entity recognition can be performed through entity recognition technology to obtain nodes at the entity granularity.

[0090] The above embodiments are described by taking the division of three granularities, namely paragraph granularity, sentence granularity, and entity granularity, as examples. It can also be any two of the above three granularities. For example, it can be paragraph granularity and sentence granularity, or sentence granularity and entity granularity, or paragraph granularity and entity granularity, etc. Of course, it can also be any other granularity, and this solution does not make specific limitations on this.

[0091] As Figure 4a shown, it is a schematic diagram of node division for a cross-paragraph text provided by an embodiment of the present application. Among them, the cross-paragraph text includes:

[0092] Query: When was the author of Harry Potter born?

[0093] Paragraph 1: Harry Potter is a series of seven fantasy novels, written by British author J.K.Rowling……

[0094] Paragraph 2: J.K.Rowling (born 31 July 1965), is best known for writing the Harry Potter fantasy series……

[0095] Referring to Figure 4a , among them, the number of nodes at the paragraph granularity is 2, the number of nodes at the sentence granularity shown in the figure is 5, and the number of nodes at the entity granularity shown in the figure is 6. The total number of nodes in Paragraph 1 is 6, and the total number of nodes in Paragraph 2 is 5.

[0096] 301B. Obtain the edges corresponding to the nodes in each paragraph according to the relationships between the nodes in each of the at least two paragraphs;

[0097] Based on the inclusion relationships between paragraphs and sentences, the inclusion relationships between sentences and entities, and the context relationships between sentences in the above cross-paragraph text T, establish the connections of edges between paragraphs and sentences within the paragraphs, between adjacent sentences within the same paragraph, between sentences and entities within the sentences, and between entities within the same sentence.

[0098] As Figure 4b shown, based on the above inclusion relationships and context relationships, etc., the edges corresponding to the nodes within Paragraph 1 and the edges corresponding to the nodes within Paragraph 2 can be obtained. Figure 4a shown

[0099] 301C. Obtain the edges corresponding to the cross-paragraph nodes according to the relationships of the nodes between the at least two paragraphs.

[0100] By fully capturing the relationships of the cross-paragraph text at various granularities, including but not limited to relationships such as web hyperlinks, containing the same entity, entity co-occurrence, entity coreference, etc., that is, sentences with web hyperlink relationships, sentences with the same entity, entities indicating the same meaning, etc. between at least two paragraphs, according to the above various relationships, further establish the connections of cross-paragraph edges at different granularities.

[0101] Among them, the multi-granularity graph of the cross-paragraph text T includes nodes corresponding to at least two paragraphs of the cross-paragraph text T, edges corresponding to the nodes within the at least two paragraphs of the cross-paragraph text T, and edges corresponding to the cross-paragraph nodes.

[0102] As Figure 4c shown, in order to obtain the edges corresponding to the cross-paragraph nodes between Paragraph 1 and Paragraph 2 on the basis of Figure 4b and further obtain the multi-granularity graph.

[0103] Based on the text organizational structure of the cross-paragraph text and the above various relationships captured in the cross-paragraph text, the above embodiments perform multi-granularity graph construction, achieving the purpose of fully expressing the connections of the cross-paragraph text at different levels of semantics.

[0104] 302. Obtain the graph node feature vector of the cross-paragraph text according to the first semantic representation vector of the cross-paragraph text and the multi-granularity graph, where the graph node feature vector indicates the semantic information of sub-texts with multiple granularities within the same paragraph and across paragraphs of the cross-paragraph text;

[0105] The graph node feature vector is usually a fixed-length real-valued vector used to represent the data or information of different nodes in the graph.

[0106] Among them, step 302 may specifically include 3021 - 3023, as follows:

[0107] 3021. Obtain a first intermediate result based on the first semantic representation vector of the cross-paragraph text and the sub-texts carried by the nodes of multiple granularities in the multi-granularity graph;

[0108] For cross-paragraph text, output a first semantic representation vector based on a deep pre-trained language model, and then obtain a first intermediate result corresponding to the nodes in the multi-granularity graph based on this first semantic representation vector. Among them, in order to distinguish nodes of different granularities (types), the graph node feature vectors of different granularity nodes are mapped to different vector spaces.

[0109] 3022. Obtain a second intermediate result based on the relationship between the first intermediate result and the sub-texts of the same granularity;

[0110] Among them, step 3022 may include steps 3022A - 3022C, specifically as follows:

[0111] 3022A. Input the first intermediate result and a first sub-graph into a first neural network to obtain the first intermediate result after the first sub-graph is updated. The first sub-graph includes the nodes of the first granularity in the multi-granularity graph and the edges corresponding to the nodes of the first granularity;

[0112] For example, the first granularity is the paragraph granularity, and the first sub-graph can be referred to Figure 4d as shown.

[0113] 3022B. Input the first intermediate result and a second sub-graph into a second neural network to obtain the first intermediate result after the second sub-graph is updated. The second sub-graph includes the nodes of the second granularity in the multi-granularity graph and the edges corresponding to the nodes of the second granularity;

[0114] For example, the second granularity is the sentence granularity, and the second sub-graph can be referred to Figure 4e as shown.

[0115] The above first neural network and second neural network can be heterogeneous graph neural networks. Among them, a heterogeneous graph neural network is a graph neural network with a heterogeneous graph as the computational graph, which distinguishes the types of nodes and edges when performing information transmission, transformation, and aggregation on the graph. A heterogeneous graph refers to a graph in which there is more than one type of node type or edge type. Specifically, different granularities correspond to heterogeneous graph neural networks with different parameters.

[0116] 3022C. Obtain the second intermediate result based on the first intermediate result after the first sub-graph is updated and the first intermediate result after the second sub-graph is updated.

[0117] By integrating the graph node feature vectors updated by the above first granularity and second granularity respectively, the second intermediate result is obtained.

[0118] Since cross-paragraph texts are related at different semantic levels and the focus of semantic relations at different levels is different, by capturing the relations at each semantic level, cross-paragraph semantic information can be extracted more comprehensively and fully.

[0119] When constructing multi-granularity graphs based on cross-paragraph texts as described above, connections of cross-paragraph edges are established according to the relations of cross-paragraph texts at each granularity. Therefore, through information interaction within nodes of the same granularity, cross-paragraph semantic relations can be fully captured at different levels of semantic information.

[0120] Embodiments of this application are described by taking two granularities as examples. Correspondingly, when the types of the above different granularities are three, for example, the third granularity is entity granularity, a sub-graph corresponding to the entity granularity is further included. The processing means are the same as those of the first granularity and the second granularity, and this solution will not be elaborated here.

[0121] 3023. Obtain the graph node feature vector of the cross-paragraph text according to the relationship between the second intermediate result and sub-texts of different types of granularities.

[0122] Specifically, the graph node feature vector is obtained by inputting the second intermediate result and the third sub-graph into a third neural network.

[0123] Among them, the third sub-graph can be referred to Figure 4f as shown. By retaining the edges between nodes of different granularities and the corresponding nodes in Figure 4c multi-granularity, the Figure 4f .

[0124] The graph node feature vector is generated by further performing information interaction between nodes of different granularities. The graph node feature vector contains semantic information at multiple levels from macroscopic to local.

[0125] Among them, the paragraph granularity node contains paragraph-level semantic information, the sentence granularity node contains sentence-level semantic information, and the entity granularity node contains entity-level semantic information. The semantic information from the paragraph level to the entity level contains information from coarse-grained to fine-grained, from macroscopic to local. After performing information interaction between nodes of different granularities on the graph node feature vector, the feature vector of each type of node in the graph contains different levels of semantic information from macroscopic to local, which makes the information contained in the graph node feature vector more comprehensive and rich.

[0126] 303. Obtain the second semantic representation vector of the cross-paragraph text. Both the first semantic representation vector and the second semantic representation vector indicate the semantic information of the cross-paragraph text. The second semantic representation vector is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector.

[0127] The second semantic representation vector of the cross-paragraph text is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector.

[0128] The above-mentioned second semantic representation vector is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector. That is to say, it is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector. This fusion is based on the first semantic representation vector of the cross-paragraph text, and fuses the relationships between sub-texts of various granularities within the same paragraph and across paragraphs of the cross-paragraph text to obtain the above-mentioned second semantic representation vector. For example, the fusion can be performed by means of vector superposition, or other means can also be used. This solution does not make specific limitations on this.

[0129] Through the embodiments of the present application, by obtaining the first semantic representation vector of the cross-paragraph text and the multi-granularity graph, the graph node feature vector of the cross-paragraph text is further obtained, and the second semantic representation vector of the cross-paragraph text is obtained according to the above-mentioned first semantic representation vector and the graph node feature vector. Compared with the semantic representation vector of the cross-paragraph text obtained only based on a single granularity of entity granularity in the prior art, this solution is based on the relationships between multi-granularity texts within the same paragraph and between multi-granularity texts across paragraphs of the cross-paragraph text, and obtains the semantic representation vector of the cross-paragraph text based on multiple granularities. Since the types of granularities involved in this solution are more, the semantic information included in generating the semantic representation vector of the cross-paragraph text is richer and more comprehensive, which helps to improve the accuracy of sorting candidate paths for subsequent searches, and also helps to improve the accuracy of extracting question-and-answer answers.

[0130] On the other hand, this solution not only performs information interaction within nodes of the same granularity on the graph node feature vector, so that the graph node feature vector can fully capture the semantic connections across paragraphs at different levels of semantic information; but also performs information interaction between nodes of different granularities on the graph node feature vector, so that the feature vector of each type of node contains different levels of semantic information from macro to local, which makes the information contained in the graph node feature vector more comprehensive and rich.

[0131] Embodiment 1

[0132] The following takes the application of the method for generating the semantic representation vector of the cross-paragraph text provided by the present application to the search scenario as an example for specific description. Among them, the embodiments of the present application can be applied to search engines, such as Baidu Search, Google Search, etc.

[0133] Refer to Figure 5 As shown, it is a schematic flowchart of a web page sorting method for a search scenario provided by an embodiment of the present application. This method may include steps 501-505, which are specifically as follows:

[0134] 501. Receive the Query input by the user;

[0135] 502. Obtain the recalled web pages according to the Query input by the user;

[0136] Specifically, based on the above-mentioned Query input by the user, the web page rough recall module recalls several candidate web pages.

[0137] Generally, if it is a general Query, the web page rough recall module will recall a candidate web page set; if it is a complex Query, the web page rough recall module will recall a candidate path set, that is, each candidate path contains at least two web pages.

[0138] 503. Confirm whether the obtained is a candidate path set;

[0139] Judge whether the web page rough recall module recalls a candidate path set. Specifically, by judging whether each element in the recall set contains a single web page or multiple web pages, it is further determined whether the obtained is a candidate path set. If it is a candidate path set, that is, corresponding to a complex search scenario, the rough recall result is input into the path fine-ranking module, and step 504 is executed to output the ranking result of the candidate paths; if it is not a candidate path set, that is, corresponding to a general search scenario, the rough recall result is input into the web page fine-ranking module, and step 505 is executed to output the ranking result of the candidate web pages.

[0140] 504. If the recalled web pages are the web pages of the candidate paths, sort the M candidate paths in the candidate path set to obtain the ranking result of the M candidate paths;

[0141] The above step 504 may include 5041 - 5046, specifically as follows:

[0142] 5041. Obtain the cross-paragraph text of each candidate path in the M candidate paths;

[0143] Specifically, the candidate path set P recalled by the web page rough recall module can be expressed as:

[0144] P = {p1, p2, …… pi, …… pM}, where i ∈ (1, M)

[0145] Among them, pi represents the path ranked at the i-th position in the web page rough recall process, and M represents that the web page rough recall module has recalled M candidate paths in total.

[0146] Each of the above paths contains multiple web pages. Preferably, take the first paragraph of each web page as the web page text, then any path pi can be expressed as:

[0147] pi = {w1, w2, …… wj, …… wK}, where j ∈ (1, K)

[0148] Among them, wj represents the web page text of the j-th web page in this path, and K represents that this path includes a total of K web pages.

[0149] By splicing the Query text with the text of each candidate path among the M candidate paths respectively, the cross-paragraph text of each candidate path among the M candidate paths is obtained. Furthermore, a set of cross-paragraph texts corresponding to the M candidate paths can be obtained:

[0150] T = {t1, t2, …… ti, …… tM}, i ∈ (1, M)

[0151] Among them, the cross-paragraph text of each candidate path can be expressed as:

[0152] ti = [Q][w1][w2]……[wj]……[wK], j ∈ (1, K)

[0153] Among them, Q represents the text of the Query.

[0154] 5042. Obtain a set of first semantic representation vectors corresponding to the M candidate paths according to the cross-paragraph text of each candidate path;

[0155] Input the cross-paragraph texts t1 - tM in the cross-paragraph text set into a text semantic generation module implemented by a deep pre-trained language model respectively to obtain a set of first semantic representation vectors corresponding to the M candidate paths:

[0156] R = {r1, r2, …… ri, …… rM}, i ∈ (1, M)

[0157] Among them, ri represents the first semantic representation vector of the i-th candidate path.

[0158] 5043. Obtain a multi-granularity graph for each candidate path among the M candidate paths according to the cross-paragraph text of each candidate path;

[0159] The embodiments of the present application are described by taking nodes of three granularities as examples, and reference can be made to Figure 6a as shown.

[0160] Specifically, reference can be made to the description of generating the multi-granularity graph in the foregoing embodiments to obtain the multi-granularity graph of each candidate path, that is, the multi-granularity graph of each cross-paragraph text, which will not be elaborated here.

[0161] 5044. Obtain the graph node feature vector of each cross-paragraph text according to the first semantic representation vector and the multi-granularity graph of each cross-paragraph text among the M cross-paragraph texts;

[0162] Specifically, when obtaining the graph node feature vectors, first, the node feature vectors of the multi-granularity graph Gi (the multi-granularity graph of the i-th candidate path) are initialized based on the first semantic representation vector ri of the candidate path; then, in order to distinguish nodes of different granularities, different linear layers are used to map the node features of different granularities to different vector spaces, where the linear layer is a neural network structure, also known as a fully connected layer, and each neuron of it is connected to all neurons of the previous layer, which can achieve linear combination or linear transformation of the input; then, as Figure 6b shown, information interaction between nodes of the same granularity is performed respectively at the paragraph granularity, sentence granularity, and entity granularity by using different heterogeneous graph neural networks; finally, as Figure 6c shown, information interaction between nodes of different granularities is achieved by using the heterogeneous graph neural network. After the above steps, the graph node feature vectors of M candidate paths can be obtained:

[0163] g = {g1, g2, …… gi, …… gM}, i ∈ (1, M)

[0164] wherein, gi represents the graph node feature vector of the i-th candidate path.

[0165] 5045. Fuse the first semantic representation vector and the graph node feature vector of each cross-paragraph text among the M cross-paragraph texts to obtain the semantic representation vector of each cross-paragraph text among the M cross-paragraph texts.

[0166] Specifically, by inputting the first semantic representation vector ri and the graph node feature vector gi of each candidate path into the fusion module and using one layer of the Transformer layer for fusion, the semantic representation vector of the cross-paragraph text corresponding to each candidate path can be output. Among them, the Transformer layer is a neural network structure that realizes feature extraction only through the attention mechanism and the linear layer. Other fusion means can also be adopted, and this solution does not make specific limitations on this.

[0167] 5046. Obtain the score of each cross-paragraph text according to the semantic representation vector of each cross-paragraph text among the M cross-paragraph texts, and sort the M cross-paragraph texts according to the score.

[0168] For example, input the semantic representation vector of the cross-paragraph text of each candidate path into the path scoring module to obtain the score of each candidate path, and then determine the sorting result of the candidate paths according to the high and low scores. For example, the semantic representation vector corresponding to the first identifier token in the cross-paragraph text of each candidate path is dimensionally reduced through a neural network, and then processed through an activation function, and then the score of each candidate path is obtained. Preferably, the candidate paths are sorted from high score to low score.

[0169] 505. If the obtained set is not the candidate path set, perform fine ranking on the recalled web pages to obtain the ranking result of the recalled web pages.

[0170] Specifically, the web page fine ranking module usually uses a deep pre-trained language model to generate text semantic vectors and scores the web pages based on the text semantic vectors. For example, the semantic representation vector corresponding to the first identifier token in the web page text is dimensionally reduced through a neural network and then processed through an activation function to obtain the score of each web page.

[0171] Through the embodiments of the present application, by constructing a multi-granularity graph for the cross-paragraph text of each candidate path, the relationship between multiple paragraphs inside the path is fully expressed, which helps to capture the semantic connection of the cross-paragraph text, extract cross-paragraph semantic information, and lay a foundation for obtaining the graph node feature vector subsequently. On the other hand, by generating the graph node feature vector based on multi-granularity information fusion, the graph node feature vector can not only represent the cross-paragraph semantic connection but also contain semantic information at different levels from macro to local, realizing the effective supplement and enhancement of the text semantic representation vector.

[0172] By fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector to obtain the semantic representation vector of the cross-paragraph text, the ranking effect is effectively improved when calculating scores for candidate paths for ranking.

[0173] Embodiment 2

[0174] The following takes the application of the method for generating the semantic representation vector of cross-paragraph text provided by the present application to the question and answer scenario as an example for specific description.

[0175] Refer to Figure 7 As shown, it is a schematic flowchart of a method for obtaining question answers provided by the embodiments of the present application. The method may include steps 701-705, which are specifically as follows:

[0176] 701. Receive the Question input by the user;

[0177] 702. Obtain evidence paragraphs according to the Question input by the user;

[0178] Specifically, based on the Question input by the user above, the evidence paragraph recall module recalls several evidence paragraphs. Among them, the evidence paragraph is a paragraph containing the necessary semantic information for extracting the answer.

[0179] Generally, if it is a general Question, the evidence paragraph recall module will recall a single web page; if it is a complex Question, the evidence paragraph recall module will recall an inference path including multiple web pages.

[0180] 703. Confirm whether the evidence paragraph belongs to the evidence paragraph of the reasoning path;

[0181] Specifically, it is determined whether the evidence paragraph recalled by the evidence paragraph recall module is a reasoning path, that is, it is determined whether the recall result contains multiple web pages. If it is a reasoning path, that is, corresponding to a complex question-and-answer scenario, the Question and the recalled reasoning path are input into the cross-paragraph answer extraction module, and step 704 is executed to output the extracted answer; if it is not a reasoning path, that is, corresponding to a general question-and-answer scenario, the Question and the recalled single evidence paragraph are input into the single-paragraph answer extraction module, and step 705 is executed to output the extracted answer.

[0182] 704. If the evidence paragraph belongs to the evidence paragraph of the reasoning path, obtain the extracted answer based on the generation method of the cross-paragraph text semantic representation vector;

[0183] Among them, step 704 is to obtain the extracted answer based on a reasoning path. Correspondingly, when M in Embodiment 1 takes the value of 1, the cross-paragraph text semantic representation vector corresponding to the evidence paragraph can be obtained based on the relevant description in step 504 of Embodiment 1. Then, a neural network is used to predict the probability that each token in the cross-paragraph text is the start and end of the answer respectively according to the semantic representation vector of each token in the cross-paragraph text. Among them, the extracted answer can be obtained according to the token with the maximum probability of being the start and end of the answer. For specific details, refer to the description in the foregoing embodiments and will not be elaborated here.

[0184] It should be noted that the evidence paragraph recall module of this solution recalls a reasoning path. The evidence paragraph recall module can use the candidate path ranking means described in Embodiment 1 to obtain the candidate path corresponding to the highest score, which is the reasoning path.

[0185] 705. If the evidence paragraph does not belong to the evidence paragraph of the reasoning path, obtain the extracted answer according to the single evidence paragraph.

[0186] Through the embodiments of the present application, by constructing a multi-granularity graph for the cross-paragraph text of the reasoning path, the relationship between multiple paragraphs inside the path is fully expressed, which helps to capture the semantic connection of the cross-paragraph text, extract the cross-paragraph semantic information, and lay a foundation for obtaining the graph node feature vector subsequently. On the other hand, by generating the graph node feature vector based on multi-granularity information fusion, the graph node feature vector can not only represent the semantic connection of the cross-paragraph, but also contain different levels of semantic information from macro to local, realizing the effective supplement and enhancement of the text semantic representation vector.

[0187] By fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector to obtain the semantic representation vector of the cross-paragraph text, the accuracy of answer extraction is improved.

[0188] Referring to Figure 8 shown in the figure, a device for generating a cross-paragraph text semantic representation vector provided by an embodiment of the present application. The device includes a first generation module 801, a second generation module 802, and a third generation module 803, which are specifically as follows:

[0189] The first generation module 801 is configured to obtain a first semantic representation vector and a multi-granularity graph of the cross-paragraph text according to the cross-paragraph text, where the cross-paragraph text includes at least two paragraphs, and the multi-granularity graph indicates the relationships between sub-texts of multiple granularities within the same paragraph in the cross-paragraph text, as well as the relationships between sub-texts of multiple granularities across paragraphs;

[0190] The second generation module 802 is configured to obtain a graph node feature vector of the cross-paragraph text according to the first semantic representation vector of the cross-paragraph text and the multi-granularity graph, and the graph node feature vector indicates the semantic information of sub-texts of multiple granularities within the same paragraph and across paragraphs in the cross-paragraph text;

[0191] The third generation module 803 is configured to obtain a second semantic representation vector of the cross-paragraph text, both the first semantic representation vector and the second semantic representation vector indicate the semantic information of the cross-paragraph text, and the second semantic representation vector is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector.

[0192] Optionally, the multi-granularity graph includes nodes of at least two granularities and edges for connecting two nodes, where each node of one of the at least two granularities of nodes indicates a sub-text of one granularity, and the edge indicates the relationship between the two connected nodes, and the relationship is the relationship between sub-texts of the same granularity or the relationship between sub-texts of different granularities.

[0193] Among them, the second generation module 802 is configured to: The second generation module is configured to: obtain a first intermediate result according to the first semantic representation vector of the cross-paragraph text and the sub-texts carried by nodes of multiple granularities in the multi-granularity graph; obtain a second intermediate result according to the first intermediate result and the relationship between sub-texts of the same granularity; obtain the graph node feature vector of the cross-paragraph text according to the second intermediate result and the relationship between sub-texts of different granularities.

[0194] Optionally, the multi-granularity graph includes a first granularity and a second granularity. The second generation module 802 is configured to: input the first intermediate result and the first sub-graph into a first neural network to obtain an updated first intermediate result of the first sub-graph, where the first sub-graph includes nodes of the first granularity in the multi-granularity graph and edges corresponding to the nodes of the first granularity; input the first intermediate result and a second sub-graph into a second neural network to obtain an updated first intermediate result of the second sub-graph, where the second sub-graph includes nodes of the second granularity in the multi-granularity graph and edges corresponding to the nodes of the second granularity; and obtain the second intermediate result according to the updated first intermediate result of the first sub-graph and the updated first intermediate result of the second sub-graph.

[0195] Optionally, the multi-granularity graph includes at least two of paragraph granularity, sentence granularity, and entity granularity.

[0196] Optionally, the relationship includes at least one of the following: inclusion, containing the same entity, entity co-reference, and web hyperlink.

[0197] Optionally, the first neural network and the second neural network are different heterogeneous graph neural networks.

[0198] For the specific implementation means of the above modules, reference may be made to the description of the foregoing embodiments, which will not be elaborated herein.

[0199] Referring Figure 9 as shown, a device for generating a cross-paragraph text semantic representation vector provided by an embodiment of the present application. As Figure 9 shown, the device 900 includes at least one processor 901, at least one memory 902, and at least one communication interface 903. The processor 901, the memory 902, and the communication interface 903 are connected through the communication bus and communicate with each other.

[0200] The processor 901 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above solutions.

[0201] The communication interface 903 is used to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0202] The memory 902 can be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or it can also be an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Compact Disc Read-Only Memory (CD-ROM) or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but not limited to this. The memory can exist independently and be connected to the processor through a bus. The memory can also be integrated with the processor.

[0203] Among them, the memory 902 is used to store the application program code for executing the above solution, and is controlled by the processor 901 to execute. The processor 901 is used to execute the application program code stored in the memory 902.

[0204] The code stored in the memory 902 can execute any of the above-provided methods for generating cross-paragraph text semantic representation vectors.

[0205] The embodiment of the present application further provides a chip system, which is applied to an electronic device; the chip system includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected by lines; the interface circuits are used to receive signals from the memory of the electronic device and send the signals to the processors, and the signals include the computer instructions stored in the memory; when the processors execute the computer instructions, the electronic device executes the method.

[0206] The embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored, and when it runs on a computer or a processor, it causes the computer or the processor to execute one or more steps in any of the above methods.

[0207] The embodiment of the present application further provides a computer program product containing instructions. When the computer program product runs on a computer or a processor, it causes the computer or the processor to execute one or more steps in any of the above methods.

[0208] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0209] In this application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural.

[0210] Those of ordinary skill in the art can understand all or part of the processes in the methods of the above embodiments. These processes can be completed by relevant hardware instructed by a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disks, or optical discs and other media that can store program codes.

[0211] As described above, the above are only the specific implementation manners of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of the present application should be covered by the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

1. A method for generating a semantic representation vector of cross-paragraph text, characterized in that, it includes: obtaining a first semantic representation vector and a multi-granularity graph of the cross-paragraph text according to the cross-paragraph text, wherein the cross-paragraph text includes at least two paragraphs, and the multi-granularity graph indicates the relationships between sub-texts of multiple granularities within the same paragraph in the cross-paragraph text, as well as the relationships between sub-texts of multiple granularities across paragraphs; obtaining a graph node feature vector of the cross-paragraph text according to the first semantic representation vector of the cross-paragraph text and the multi-granularity graph, the graph node feature vector indicating the semantic information of sub-texts of multiple granularities within the same paragraph and across paragraphs in the cross-paragraph text; obtaining a second semantic representation vector of the cross-paragraph text, both the first semantic representation vector and the second semantic representation vector indicating the semantic information of the cross-paragraph text, and the second semantic representation vector is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector.

2. The method according to claim 1, characterized in that, the multi-granularity graph includes nodes of at least two granularities and edges for connecting two nodes, wherein each node of one of the at least two granularities of nodes indicates a sub-text of one granularity, and the edge indicates the relationship between the two connected nodes, and the relationship is the relationship between sub-texts of the same granularity or the relationship between sub-texts of different granularities.

3. The method according to claim 2, characterized in that, the obtaining the graph node feature vector of the cross-paragraph text according to the first semantic representation vector of the cross-paragraph text and the multi-granularity graph includes: obtaining a first intermediate result according to the first semantic representation vector of the cross-paragraph text and the sub-texts carried by nodes of multiple granularities in the multi-granularity graph; obtaining a second intermediate result according to the first intermediate result and the relationship between sub-texts of the same granularity; obtaining the graph node feature vector of the cross-paragraph text according to the second intermediate result and the relationship between sub-texts of different granularities.

4. The method according to claim 3, characterized in that, the multi-granularity graph includes a first granularity and a second granularity, and the obtaining the second intermediate result according to the first intermediate result and the relationship between sub-texts of the same granularity includes: inputting the first intermediate result and a first sub-graph into a first neural network to obtain an updated first intermediate result of the first sub-graph, the first sub-graph including nodes of the first granularity in the multi-granularity graph and the edges corresponding to the nodes of the first granularity; inputting the first intermediate result and a second sub-graph into a second neural network to obtain an updated first intermediate result of the second sub-graph, the second sub-graph including nodes of the second granularity in the multi-granularity graph and the edges corresponding to the nodes of the second granularity; obtaining the second intermediate result according to the updated first intermediate result of the first sub-graph and the updated first intermediate result of the second sub-graph.

5. The method according to any one of claims 1 to 4, characterized in that, The multi-granularity graph includes at least two of paragraph granularity, sentence granularity, and entity granularity.

6. The method according to any one of claims 1 to 5, wherein, the relationship includes at least one of the following: containment, containing the same entity, entity co-reference, web hyperlink.

7. The method according to claim 4, wherein, the first neural network and the second neural network are different heterogeneous graph neural networks.

8. A device for generating a cross-paragraph text semantic representation vector, wherein, it includes: A first generation module, configured to obtain a first semantic representation vector and a multi-granularity graph of the cross-paragraph text according to the cross-paragraph text, wherein the cross-paragraph text includes at least two paragraphs, and the multi-granularity graph indicates the relationships between sub-texts of multiple granularities within the same paragraph in the cross-paragraph text, as well as the relationships between sub-texts of multiple granularities across paragraphs; A second generation module, configured to obtain a graph node feature vector of the cross-paragraph text according to the first semantic representation vector of the cross-paragraph text and the multi-granularity graph, and the graph node feature vector indicates the semantic information of sub-texts of multiple granularities within the same paragraph and across paragraphs in the cross-paragraph text; A third generation module, configured to obtain a second semantic representation vector of the cross-paragraph text, both the first semantic representation vector and the second semantic representation vector indicate the semantic information of the cross-paragraph text, and the second semantic representation vector is obtained by fusing the first semantic representation vector of the cross-paragraph text and the graph node feature vector.

9. The device according to claim 8, wherein, the multi-granularity graph includes nodes of at least two granularities and edges for connecting two nodes, wherein each node of one of the at least two granularities of nodes indicates a sub-text of one granularity, and the edge indicates the relationship between the two connected nodes, and the relationship is the relationship between sub-texts of the same granularity or the relationship between sub-texts of different granularities.

10. The device according to claim 9, wherein, the second generation module is configured to: Obtain a first intermediate result according to the first semantic representation vector of the cross-paragraph text and the sub-texts carried by nodes of multiple granularities in the multi-granularity graph; Obtain a second intermediate result according to the first intermediate result and the relationship between sub-texts of the same granularity; Obtain the graph node feature vector of the cross-paragraph text according to the second intermediate result and the relationship between sub-texts of different granularities.

11. The device according to claim 10, wherein, the multi-granularity graph includes a first granularity and a second granularity, and the second generation module is configured to: Input the first intermediate result and a first sub-graph into a first neural network to obtain an updated first intermediate result of the first sub-graph, where the first sub-graph includes nodes of the first granularity in the multi-granularity graph and the edges corresponding to the nodes of the first granularity; Input the first intermediate result and the second sub-graph into a second neural network to obtain the updated first intermediate result of the second sub-graph. The second sub-graph includes nodes of a second granularity in the multi-granularity graph and edges corresponding to the nodes of the second granularity. Obtain the second intermediate result based on the updated first intermediate result of the first sub-graph and the updated first intermediate result of the second sub-graph.

12. The apparatus according to any one of claims 8 to 11, wherein, the multi-granularity graph includes at least two of paragraph granularity, sentence granularity, and entity granularity.

13. The apparatus according to any one of claims 8 to 12, wherein, the relationship includes at least one of the following: containment, containing the same entity, entity co-reference, web hyperlink.

14. The apparatus according to claim 11, wherein, the first neural network and the second neural network are different heterogeneous graph neural networks.

15. A device for generating a cross-paragraph text semantic representation vector, wherein, it includes a processor and a memory; wherein, the memory is used to store program code, and the processor is used to call the program code to execute the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, wherein, the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 7.

17. A computer program product, wherein, when the computer program product runs on a computer, it causes the computer to execute the method according to any one of claims 1 to 7.

Images