Description text generation method, device and electronic device based on knowledge graph

Abstract

The embodiment of the present invention provides a method, device and electronic device for generating descriptive text based on a knowledge graph, the method comprising: obtaining a knowledge graph; extracting graph structure relationship features from multiple different angles from the knowledge graph to generate first graph embedding data; performing natural language-based decoding processing on the first graph embedding data to generate a descriptive text. The embodiment of the present invention uses the knowledge graph as input data to extract graph structure relationship features in the knowledge graph, and then generates a text description based on natural language through encoding and decoding processing of a machine learning model. In the above-mentioned natural language conversion process, the encoding and decoding processing is driven based on the graph structure relationship in the knowledge graph, so that the main points of the content in the knowledge graph can be fully captured, and a natural language description that can more fully reflect the main content of the knowledge graph is generated.

Description

Technical Field

[0001] The present application relates to a method, device and electronic device for generating descriptive text based on a knowledge graph, and belongs to the field of computer technology. Background Art

[0002] In real life and commercial applications, it is often necessary to describe some structured data in natural language so that people can quickly understand the meaning of these data. With the development of knowledge graph technology, a large amount of information is stored in the database in the form of knowledge graphs. When it is necessary to know certain information content, it is necessary to convert these knowledge graph data into natural language description documents, so that users can quickly understand the information content.

[0003] In the prior art, in the process of converting knowledge graph data into natural language, the information in the knowledge graph is generally converted into an information sequence, and then a sequence-to-sequence text generation model is used to generate the description text. This generation method will lose a lot of information and cannot obtain an ideal text description. Summary of the invention

[0004] Embodiments of the present invention provide a method, device and electronic device for generating descriptive text based on a knowledge graph, so as to generate richer text description content based on the knowledge graph.

[0005] In order to achieve the above object, an embodiment of the present invention provides a method for generating description text based on a knowledge graph, comprising:

[0006] Get the knowledge graph;

[0007] Extracting graph structure relationship features from multiple different angles from the knowledge graph to generate first graph embedding data;

[0008] The first image embedded data is subjected to natural language-based decoding processing to generate a description text.

[0009] The embodiment of the present invention also provides a description text generation device based on a knowledge graph, comprising:

[0010] Graph data acquisition module, used to obtain knowledge graphs;

[0011] An encoder module, used to extract graph structure relationship features from multiple different angles from the knowledge graph to generate first graph embedding data;

[0012] The decoder module is used to perform natural language-based decoding processing on the first image embedding data to generate a description text.

[0013] An embodiment of the present invention further provides an electronic device, including:

[0014] Memory, used to store programs;

[0015] A processor is used to run the program stored in the memory to execute the aforementioned knowledge graph-based description text generation method.

[0016] The embodiment of the present invention uses the knowledge graph as input data, extracts the graph structure relationship features in the knowledge graph, and then generates a text description based on natural language through the encoding and decoding processing of the machine learning model. In the above-mentioned natural language conversion process, the encoding and decoding processing is driven based on the graph structure relationship in the knowledge graph, so that the content points in the knowledge graph can be fully captured, and a natural language description that can more fully reflect the main content of the knowledge graph is generated.

[0017] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic diagram of an exemplary model framework structure of an embodiment of the present invention;

[0019] Figure 2 Schematic diagram of the model structure of a multi-graph convolutional neural network layer according to an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of graph structure relationship feature processing of a graph encoder according to an embodiment of the present invention;

[0021] Figure 4 A schematic diagram of a knowledge graph retrieval example according to an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of a flow chart of a method for generating description text based on a knowledge graph according to an embodiment of the present invention;

[0023] Figure 6 It is a structural schematic diagram of a description text generation device based on a knowledge graph according to an embodiment of the present invention;

[0024] Figure 7 FIG. 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0026] With the development of computer technology, all walks of life have accumulated a large amount of structured information and data, among which structured data in the form of knowledge graphs are widely used. Knowledge graphs are essentially semantic networks, a graph-based data structure consisting of points and edges. In a knowledge graph, each point represents an entity that exists in the real world, and each edge is a relationship between entities. Knowledge graphs are the most effective way to represent relationships. Knowledge graphs can connect various types of information (heterogeneous information) together to obtain a relationship network. Knowledge graphs provide the ability to analyze problems from a relationship perspective, and therefore, are also adopted by many search engines or big data analysis platforms.

[0027] Since knowledge graphs are structured data forms, and in actual applications, for most users, it is often more convenient and easier to understand information through natural language. Therefore, there is a need to convert knowledge graphs into natural language descriptions.

[0028] The technology of the embodiment of the present invention provides a technology for converting structured data such as knowledge graph into a description using natural language. In the embodiment of the present invention, the knowledge graph is used as input data, the graph structure relationship features in the knowledge graph are extracted, and then the encoding and decoding processing of the machine learning model is used to generate a text description based on natural language. In the above-mentioned natural language conversion process, the encoding and decoding processing is driven based on the graph structure relationship in the knowledge graph, so that the content points in the knowledge graph can be fully captured, and a natural language description that can more fully reflect the main content of the knowledge graph is generated.

[0029] like Figure 1 As shown in FIG. 1 , it is a schematic diagram of an exemplary model framework structure of an embodiment of the present invention. In this example, the entire model adopts an encoder-decoder architecture, such as Figure 1 As shown in the figure, the left part is the encoder model, which is mainly used to extract features and encode the input knowledge graph, generate graph embedding data containing graph structure relationship features, and then output it to the decoder model on the right. The decoder model on the right is mainly used to decode the graph embedding data to generate description text and output it.

[0030] Encoder part

[0031] In the embodiment of the present invention, a multi-graph convolutional network (MGCN) model structure is used as an example encoder model, wherein the structure of each layer can be further described as follows: Figure 2 As shown, Figure 2 The schematic diagram of the model structure of the multi-graph convolutional neural network layer of the embodiment of the present invention is shown in the figure. As shown in the figure, each multi-graph convolutional neural network layer includes multiple graph encoders. The graph embedding data after the encoding processing by the multiple graph encoders is aggregated by the aggregation layer as the output graph embedding data. As an example, in the embodiment of the present invention, the multi-graph convolutional neural network layer adopts 6 graph encoders, whose functions correspond to Figure 3 shown.

[0032] Figure 3 Schematic diagram of graph structure relationship feature processing of the graph encoder of an embodiment of the present invention. The leftmost input graph corresponds to the knowledge graph, as shown in the figure, which contains four points E1, E2, E3, and E4 and three edges R1, R2, and R3. The knowledge graph will be input into Figure 2 The encoders 1 to 6 shown in the figure are respectively performed as follows Figure 3 The processing corresponding to each graph encoder is shown. In an embodiment of the present invention, in the process of feature extraction and encoding, the edges in the knowledge graph are treated as points, so as to retain more structural relationship information in the knowledge graph and reduce the number of model parameters. For the convenience of description, the original points in the knowledge graph, that is, the points corresponding to the entities, are called first-type points, and the edges in the knowledge graph, that is, the relationships between entities, are called first-type points. In addition, in order to extract more information in the knowledge graph as a whole, a global point is added. Figure 3 In , points E1 to E4 correspond to the first type of points, points R1 to R3 are the second type of points, and point G corresponds to the global point. Each graph encoder performs the following processing:

[0033] The processing performed by the graph encoder 1 is: obtaining each first type point and second type point of the knowledge graph, and adding a self-loop edge to each first type point, second type point and global point, such as Figure 3 As shown in encoder 1 in the middle figure, each point has an edge with itself as the starting point and the end point.

[0034] The process performed by the encoder 2 is: obtaining the positive association relationship between each first type point and the second type point as the edge from the first type point to the second type point. Figure 3As shown in the encoder 2 in the figure, based on the structural relationship of the knowledge graph, positive edges are formed between points E1 to E4 and points R1 to R3, which reflect the positive association between points E1 to E4 and points R1 to R3 in the knowledge graph. For example, in the knowledge graph, edge R1 starts from point E1, so encoder 2, which extracts positive point-edge associations, generates an edge from point E1 to point R1 in its generated graph embedding data.

[0035] The process performed by the encoder 3 is: obtaining the reverse association relationship between each first type point and the second type point as the edge from the second type point to the first type point. Figure 3 As shown in the encoder 3 in the figure, based on the structural relationship of the knowledge graph, reverse edges are formed between points E1 to E4 and points R1 to R3, which reflect the reverse association relationship between points E1 to E4 and points R1 to R3 in the knowledge graph. For example, in the knowledge graph, edge R1 starts from point E1, so encoder 3, which extracts the reverse point-edge association relationship, generates an edge from point R1 to point E1 in its generated graph embedding data.

[0036] The process performed by the encoder 4 is: obtaining the positive association relationship between the first type points as the positive edge between the first type points. Figure 3 As shown in the encoder 4, based on the structural relationship of the knowledge graph, positive edges are formed between points E1 and E4, which reflect the positive association relationship between points E1 and E4 in the knowledge graph. For example, in the knowledge graph, there are edges starting from point E1 pointing to points E2 and E4. Therefore, as an encoder 4 for extracting positive association relationships between points, it generates an edge starting from point E1 pointing to points E2 and E4 in its generated graph embedding data.

[0037] The process performed by the encoder 5 is: obtaining the reverse association relationship between each first type point as the reverse edge between the first type points. Figure 3 As shown in the encoder 5, based on the structural relationship of the knowledge graph, reverse edges between points E1 to E4 are formed, and these edges reflect the reverse association relationship between points E1 to E4 in the knowledge graph. For example, in the knowledge graph, there are edges starting from point E1 and pointing to points E2 and E4. Therefore, as an encoder 5 for extracting the reverse point-to-point association relationship, it generates an edge from point E2 and point E4 to point E1 in its generated graph embedding data.

[0038] The processing performed by the graph encoder 6 is: obtaining the association relationship between the global point and the first type point and the second type point as the edge between the global point and the first type point and the second type point. The encoder 6 is used to form the association relationship between the global point and all the first type points and the second type points. Therefore, in the generated graph embedding data, edges are generated from the global point G to points E1 to E4 and points R1 to R3 respectively.

[0039] The above-mentioned multiple graph encoders extract and encode the graph structure relationship features of the knowledge graph from different angles. Such a simple and clear conversion process has greatly helped the learning in the multi-graph convolutional neural network, thereby enabling in-depth and comprehensive mining of the information content contained in the knowledge graph.

[0040] Figure 2 The various multi-graph convolutional neural network layers shown can be implemented using the same model structure as follows Figure 3 The input knowledge graph is first encoded for graph embedding to generate initial graph embedding data, which is then input to the first layer of the multi-graph convolutional neural network. The graph embedding data generated after the first layer of the multi-graph convolutional neural network is further processed by the next layer of the multi-graph convolutional neural network until the nth layer of the multi-graph convolutional neural network. Figure 2 As shown in , the input graph embedding data can be the initial graph embedding data or the graph embedding data from the previous layer, and the output graph embedding data also serves as the input graph embedding data of the next layer. The graph embedding data output by each layer of multi-graph convolutional neural network processing will be superimposed and provided to Figure 1 The decoder model shown.

[0041] By encoding graph features through the model structure of a multi-layer multi-graph convolutional neural network, deeper abstract information can be captured. The specific number of layers n can be determined according to the actual model training results to obtain more suitable text description results.

[0042] Decoder part

[0043] The decoder module in this embodiment can adopt a decoder model based on natural language processing, so that the graph embedding information output by the encoder model can be converted into natural language encoding for output. As an example, Figure 1As shown in the right part, the decoder model mainly includes multiple LSTM layers and a cross-attention layer. The output word embedding vector is input into the LSTM layer of the first layer, and then processed by multiple LSTM layers and the graph embedding data after the encoder model output is superimposed, and then input into the cross-attention layer for processing, and finally the encoding of the description text is output, and finally it can be converted into a description text. The number of LSTM layers m in the figure can be determined according to actual needs and model training effects. The decoder model can adopt a model that does not limit the length of the output text, so that the output text can be more flexible and more accurately describe the key content information contained in the knowledge graph.

[0044] The above model adopts an encoder-decoder text generation model, in which a multi-graph convolutional neural network is used to extract and encode the graph structure relationship features of the knowledge graph, thereby effectively avoiding the problems of information loss and excessive parameters, selectively capturing important information in multiple knowledge graphs and effectively aggregating them, so that the generated description text can more effectively describe the key information contained in the knowledge graph. The embodiment of the present invention proposes the above model processing, which does not require a strict correspondence between input and output, so that richer description content can be generated with the assistance of knowledge graphs.

[0045] Acquisition of knowledge graph and application of models

[0046] In an embodiment of the present invention, the knowledge graph used as the input of the above-mentioned model can come from the search results performed in the database of the knowledge graph. With the development of big data technology, a large amount of knowledge graph data has been accumulated in various industries and fields, and these knowledge graph data can provide users with keyword-based information queries. In some application scenarios, users can carry query keywords through query requests, and search in the knowledge graph database based on these keywords to obtain relevant knowledge graphs, and input the retrieved knowledge graph into the model provided in the embodiment of the present invention, output the description text, and present it to the user.

[0047] In the process of acquiring the knowledge graph, the knowledge graph can be acquired based on the main entity and the topic-related entity, and then the text description of the topic can be generated using the model provided by the embodiment of the present invention. Specifically, the main entity refers to the core execution of the information to be acquired, and the topic-related entity refers to some subject content associated with the main entity. The main entity and the topic-related entity correspond to the keywords in the specific retrieval process, which are explained below through an example.

[0048] like Figure 4 As shown, it is a schematic diagram of a retrieval example of a knowledge graph in an embodiment of the present invention. As the search keyword, the user provides the name of a singer "AAAA" as the keyword corresponding to the main entity, as well as "retro style", "funk", "rhythm and blues", etc. In an embodiment of the present invention, the retrieval process based on the main entity and the subsidiary subject entity can follow the following strategy: obtain the first triple directly associated with the main entity and the second triple that associates the main entity with the subsidiary subject entity. The knowledge graph can generally be recorded in the form of triples, and the more typical triple form is entity-relationship-entity, where the entity corresponds to the point in the knowledge graph and the relationship corresponds to the edge in the knowledge graph. In the actual retrieval process, the main entity and the subsidiary subject entity will be located in the knowledge graph database first, and then the corresponding triples will be extracted according to the above strategy to form a knowledge graph as the retrieval result. The data stored in the above-mentioned knowledge graph database can be considered as one or more larger knowledge graphs, and the knowledge graph as the retrieval result is only a part of the large knowledge graph in the database.

[0049] like Figure 4 As shown, the acquired knowledge graph includes the main entity located in the center and multiple subsidiary entities 1 to 4 and other entities 5 to 10 other than the main entity and the subsidiary entities. Among them, entities 2, 3, 5, 6, 7, 8, and 9 are entities directly associated with the main entity, and entities 2 and 3 are also subsidiary subject entities. The triples formed by these entities and the main entity are the first triples mentioned above. In addition, entities 1, 3, and 4 are subsidiary subject entities. They are not directly associated with the main entity. They are associated with the main entity through entities 9 and 10, entity 8, and entity 7 respectively. The triples formed on these association paths are the second triples mentioned above. In the second triple, there are some triples that overlap with the first triple, such as the triples formed between entity 8, entity 7, and entity 9 and the main entity. Since we ultimately want to obtain a knowledge graph, there is no problem of triple duplication. The above division of triples into the first triple and the second triple is only for the convenience of description.

[0050] Back to the example of taking the name of a singer "AAAA" as a subsidiary subject entity related to the main entity, by searching the knowledge graph database, we can obtain, for example, Figure 4 The knowledge graph shown in the figure can then be input into the aforementioned Figure 1The model shown in the figure is used for processing, so that a text description based on the main entity related to the singer and the subsidiary subject entity can be generated. For example, the text description obtained is "the singer AAAA was born on October 10, 1985. He is a singer-songwriter, music producer, and dancer. He is well-known for his stage performances. He can perform a wide range of musical styles including retro, funk, rhythm and blues, R&B, hip pop, and rock."

[0051] The method and machine learning model provided in the embodiment of the present invention can be applied or deployed on a cloud platform that provides data services, such as a search engine that provides retrieval services, an e-commerce service platform, etc., to combine information retrieval with text descriptions in natural language. Of course, the above-mentioned machine learning model can also be deployed on a local computer, mobile terminal or server to provide conversion processing based on knowledge graphs to natural language descriptions.

[0052] In terms of application scenarios, it can be applied in many aspects such as business, life, and information acquisition. Taking the e-commerce platform as an example, you can search the knowledge graph database related to the product based on the keywords provided by the user. After obtaining the knowledge graph, it is converted into a natural language description of the product and provided to the user. Among the keywords provided by the user, the product name can be used as the main entity, and other attribute keywords can be used as subsidiary subject entities. The knowledge graph can be retrieved and obtained using the already stimulated e-commerce knowledge base. In addition, it can also be applied to the data officer project. According to the knowledge graph related to the operational data accumulated in a certain field, the knowledge graph can be retrieved, and a text description of the operational status or sales status of the field can be generated.

[0053] The above-mentioned machine learning model can use the data in the existing knowledge graph database and the existing corresponding text descriptions as training data. For example, the knowledge graph data accumulated in the knowledge encyclopedia database and the corresponding text descriptions can be used as training data. In addition, for some professional fields, such as the e-commerce industry, the accumulated knowledge graph data of various products sold by e-commerce and the corresponding descriptions on the product pages can also be used as training data.

[0054] The technical solution of the present invention is further illustrated by some specific embodiments below.

[0055] Embodiment 1

[0056] like Figure 5 As shown, it is a flow chart of a method for generating description text based on a knowledge graph according to an embodiment of the present invention, and the method includes:

[0057] S101: Obtaining a knowledge graph. The knowledge graph may be obtained from a knowledge graph database. Specifically, after obtaining a user query request, the process of obtaining the knowledge graph may obtain from the query request a first keyword corresponding to the main entity and a second keyword corresponding to the subsidiary subject entity, and then search the knowledge graph database according to the first keyword and the second keyword to obtain the knowledge graph, wherein the knowledge graph includes a first triple directly associated with the main entity and a second triple that associates the main entity with the subsidiary subject entity.

[0058] S102: Extracting graph structure relationship features from multiple different angles from the knowledge graph to generate first graph embedding data. The specific feature extraction method may be: treating the edges in the knowledge graph as points, extracting graph structure relationship features from the knowledge graph, generating multiple second graph embedding data from different angles, and then aggregating the multiple second graph embedding data to generate the first graph embedding data.

[0059] Specifically, the above-mentioned extraction of graph structure relationship features at multiple different angles may include one or more combinations of the following aspects of feature extraction and encoding processing. The detailed examples of this part can be as follows: Figure 3 The processing mechanism is shown:

[0060] 1) Obtain each first-type point and second-type point of the knowledge graph, and add a self-loop edge to each first-type point, second-type point and global point to generate second graph embedding data.

[0061] 2) Obtain the positive association relationship between each first type point and the second type point as the edge from the first type point to the second type point, and generate the second graph embedding data.

[0062] 3) Obtain the reverse association relationship between each first type point and the second type point as the edge from the second type point to the first type point, and generate second graph embedding data.

[0063] 4) Obtain the positive association relationship between the first type points as the positive edge between the first type points, and generate the second graph embedding data.

[0064] 5) Obtain the reverse association relationship between each first type point as the reverse edge between the first type points, and generate second graph embedding data.

[0065] 6) Obtain the association relationship between the global point and the first type of point and the second type of point as the edge between the global point and the first type of point and the second type of point, and generate second graph embedding data.

[0066] Specifically, the processing performed in this step can be implemented using a multi-layer multi-graph convolutional neural network. Accordingly, this step can specifically include:

[0067] The knowledge graph is encoded for graph embedding to generate initial graph embedding data. Through the multi-layer multi-graph convolutional neural network, the following layer-by-layer processing is performed:

[0068] Treat the edges in the knowledge graph as points, extract the graph structure relationship features of the knowledge graph, and generate second graph embedding data from multiple different angles;

[0069] Aggregate the multiple second graph embedding data to generate third graph embedding data, and repeat the above process as the initial graph embedding data of the next layer of multi-graph convolutional neural network until the last layer of multi-graph convolutional neural network is reached;

[0070] The third-image embedding data output by each layer of the multi-image convolutional neural network is superimposed and used as the first-image embedding data.

[0071] The multi-graph convolutional neural network of each layer mentioned above can include the aforementioned Figure 2 The multiple graph encoders shown in the figure each correspond to the encoding processing of 1) to 6) above. The specific model structure of the multi-layer multi-graph convolutional neural network can refer to the previously introduced Figure 1 and Figure 2 shown.

[0072] S103: Perform natural language decoding processing on the first image embedding data to generate a description text. The decoding part can be performed using a long short-term memory network and a cross attention model to perform natural language decoding processing on the image embedding data with no length limit to generate a description text.

[0073] The embodiment of the present invention uses the knowledge graph as input data, extracts the graph structure relationship features in the knowledge graph, and then generates a text description based on natural language through the encoding and decoding processing of the machine learning model. In the above-mentioned natural language conversion process, the encoding and decoding processing is driven based on the graph structure relationship in the knowledge graph, so that the content points in the knowledge graph can be fully captured, and a natural language description that can more fully reflect the main content of the knowledge graph is generated.

[0074] Embodiment 2

[0075] like Figure 6 As shown, it is a schematic diagram of the structure of a description text generation device based on a knowledge graph according to an embodiment of the present invention, and the device includes:

[0076] The graph data acquisition module 11 is used to acquire the knowledge graph. The knowledge graph can be acquired from the knowledge graph database. Specifically, after acquiring the user query request, the process of acquiring the knowledge graph can acquire the first keyword corresponding to the main entity and the second keyword corresponding to the subsidiary subject entity from the query request, and then search the knowledge graph database according to the first keyword and the second keyword to acquire the knowledge graph, wherein the knowledge graph includes the first triple directly associated with the main entity and the second triple that associates the main entity with the subsidiary subject entity.

[0077] The encoder module 12 is used to extract graph structure relationship features of multiple different angles from the knowledge graph to generate first graph embedding data. The specific feature extraction method can be: treating the edges in the knowledge graph as points, extracting graph structure relationship features from the knowledge graph, generating multiple second graph embedding data from different angles, and then aggregating the multiple second graph embedding data to generate the first graph embedding data.

[0078] Specifically, the above-mentioned extraction of graph structure relationship features at multiple different angles may include one or more combinations of the feature extraction and encoding processes shown in 1) to 6) in the first embodiment.

[0079] Furthermore, the processing performed by the encoder module 12 may be implemented using a multi-layer multi-graph convolutional neural network, which may specifically include:

[0080] The knowledge graph is encoded for graph embedding to generate initial graph embedding data. Through the multi-layer multi-graph convolutional neural network, the following layer-by-layer processing is performed:

[0081] Treat the edges in the knowledge graph as points, extract the graph structure relationship features of the knowledge graph, and generate second graph embedding data from multiple different angles;

[0082] Aggregate the multiple second graph embedding data to generate third graph embedding data, and repeat the above process as the initial graph embedding data of the next layer of multi-graph convolutional neural network until the last layer of multi-graph convolutional neural network is reached;

[0083] The third-image embedding data output by each layer of the multi-image convolutional neural network is superimposed and used as the first-image embedding data.

[0084] The multi-graph convolutional neural network of each layer mentioned above can include the aforementioned Figure 2 The multiple graph encoders shown in the figure each correspond to the encoding processing of 1) to 6) above. The specific model structure of the multi-layer multi-graph convolutional neural network can refer to the previously introduced Figure 1 and Figure 2 shown.

[0085] The decoder module 13 is used to perform natural language based decoding processing on the first image embedding data to generate a description text. The decoding part can be performed using a long short-term memory network and a cross attention model to perform natural language based decoding processing on the image embedding data with no length limit to generate a description text.

[0086] The specific description of the above-mentioned processing process, the detailed description of the technical principle and the detailed analysis of the technical effect have been described in detail in the previous embodiments and will not be repeated here.

[0087] Embodiment 3

[0088] The above embodiments describe the process processing and device structure of the description text generation method based on the knowledge graph of the embodiment of the invention. The functions of the above method and device can be implemented with the help of an electronic device, such as Figure 7 As shown, it is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, which specifically includes: a memory 110 and a processor 120.

[0089] The memory 110 is used to store programs.

[0090] In addition to the above-mentioned programs, the memory 110 may also be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device, contact data, phone book data, messages, pictures, videos, etc.

[0091] The memory 110 may be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0092] The processor 120 is coupled to the memory 110 and is used to execute the program in the memory 110 to perform the operating steps of the description text generation method based on the knowledge graph described in the above embodiment.

[0093] In addition, the processor 120 may also include the various modules described in the aforementioned embodiments to perform the processing of descriptive text generation based on the knowledge graph, and the memory 110 may be used, for example, to store the data required for these modules to perform operations and / or the output data.

[0094] The specific description of the above-mentioned processing process, the detailed description of the technical principle and the detailed analysis of the technical effect have been described in detail in the previous embodiments and will not be repeated here.

[0095] Furthermore, as shown in the figure, the electronic device may also include other components such as a communication component 130, a power component 140, an audio component 150, and a display 160. The figure only schematically shows some components, which does not mean that the electronic device only includes the components shown in the figure.

[0096] The communication component 130 is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component 130 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 130 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0097] The power supply component 140 provides power to various components of the electronic device. The power supply component 140 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device.

[0098] The audio component 150 is configured to output and / or input audio signals. For example, the audio component 150 includes a microphone (MIC), and when the electronic device is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 110 or sent via the communication component 130. In some embodiments, the audio component 150 also includes a speaker for outputting audio signals.

[0099] The display 160 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0100] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

[0101] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating description text based on knowledge graph, comprising: Get the knowledge graph; The knowledge graph is subjected to a graph embedding encoding process to generate initial graph embedding data, and the following layer-by-layer processing is performed through a multi-layer multi-graph convolutional neural network: the edges in the knowledge graph are regarded as points, and the graph structure relationship features of the knowledge graph are extracted to generate a plurality of second graph embedding data at different angles; the plurality of second graph embedding data are aggregated to generate third graph embedding data, which is used as the initial graph embedding data of the next layer of the multi-graph convolutional neural network; the above processing is repeated until the last layer of the multi-graph convolutional neural network is reached; the third graph embedding data output by each layer of the multi-graph convolutional neural network is superimposed and used as the first graph embedding data; The first image embedding data is decoded based on natural language through a long short-term memory network and a cross attention model to generate a description text.

2. The method according to claim 1, wherein: In the second graph embedding data, the edges in the knowledge graph are regarded as points, the points corresponding to the entities are regarded as the first type of points, the points corresponding to the edges are regarded as the second type of points, and the global points are added. Accordingly, extracting graph structure relationship features from multiple different angles from the knowledge graph to generate first graph embedding data includes: Obtain each of the first type points and the second type points of the knowledge graph, and add a self-loop edge to each of the first type points and the second type points and the global point to generate the second graph embedding data; and, Acquire the positive association relationship between each of the first type points and the second type points as the edge from the first type point to the second type point, and generate the second graph embedding data; and, Obtaining a reverse association relationship between each of the first type points and the second type points as an edge from the second type point to the first type point, and generating the second graph embedding data; and, Acquire positive associations between the first-type points as positive edges between the first-type points, and generate the second graph embedding data; and, Obtaining reverse association relationships between the first-type points as reverse edges between the first-type points, and generating the second graph embedding data; and, The association relationship between the global point and the first type of point and the second type of point is obtained as an edge between the global point and the first type of point and the second type of point, and the second graph embedding data is generated.

3. The method according to claim 1, wherein: The knowledge graph includes a first triple and a second triple, and acquiring the knowledge graph includes: Obtaining a query request, wherein the query request includes a first keyword corresponding to a main entity and a second keyword corresponding to a subsidiary subject entity; According to the first keyword and the second keyword, a search is performed in the knowledge graph database to obtain the knowledge graph, wherein the knowledge graph includes a first triplet directly associated with a main entity and a second triplet that associates the main entity with the subsidiary subject entity.

4. The method according to claim 1, wherein: The step of performing natural language-based decoding processing on the first image embedded data to generate a description text includes: The long short-term memory network and the cross attention model are used to perform natural language decoding processing on the graph embedding data with unlimited length to generate a description text.

5. A description text generation device based on knowledge graph, comprising: Graph data acquisition module, used to obtain knowledge graphs; The encoder module is used to perform graph embedding encoding processing on the knowledge graph to generate initial graph embedding data, and perform the following layer-by-layer processing through a multi-graph convolutional neural network: the edges in the knowledge graph are regarded as points, and the graph structure relationship features of the knowledge graph are extracted to generate a plurality of second graph embedding data at different angles; the plurality of second graph embedding data are aggregated to generate third graph embedding data as the initial graph embedding data of the next layer of the multi-graph convolutional neural network; the above processing is repeated until the last layer of the multi-graph convolutional neural network is reached; the third graph embedding data output by each layer of the multi-graph convolutional neural network is superimposed and used as the first graph embedding data; The decoder module is used to perform natural language-based decoding processing on the first image embedding data through a long short-term memory network and a cross attention model to generate a description text.

6. The device according to claim 5, wherein: The knowledge graph includes a first triple and a second triple, and acquiring the knowledge graph includes: Obtaining a query request, wherein the query request includes a first keyword corresponding to a main entity and a second keyword corresponding to a subsidiary subject entity; According to the first keyword and the second keyword, a search is performed in the knowledge graph database to obtain the knowledge graph, wherein the knowledge graph includes a first triplet directly associated with a main entity and a second triplet that associates the main entity with the subsidiary subject entity.

7. The device according to claim 5, wherein: In the second graph embedding data, the edges in the knowledge graph are regarded as points, the points corresponding to the entities are regarded as the first type of points, the points corresponding to the edges are regarded as the second type of points, and the global points are added. Accordingly, extracting graph structure relationship features from multiple different angles from the knowledge graph to generate first graph embedding data includes: Obtain each of the first type points and the second type points of the knowledge graph, and add a self-loop edge to each of the first type points and the second type points and the global point to generate the second graph embedding data; and, Acquire the positive association relationship between each of the first type points and the second type points as the edge from the first type point to the second type point, and generate the second graph embedding data; and, Obtaining a reverse association relationship between each of the first type points and the second type points as an edge from the second type point to the first type point, and generating the second graph embedding data; and, Acquire positive associations between the first-type points as positive edges between the first-type points, and generate the second graph embedding data; and, Obtaining reverse association relationships between the first-type points as reverse edges between the first-type points, and generating the second graph embedding data; and, The association relationship between the global point and the first type of point and the second type of point is obtained as an edge between the global point and the first type of point and the second type of point, and the second graph embedding data is generated.

8. An electronic device, comprising: Memory, used to store programs; A processor, used to run the program stored in the memory to execute the description text generation method based on the knowledge graph as described in any one of claims 1 to 4.

Images