Knowledge question-answering method and device based on vehicle information data

Abstract

The invention discloses a knowledge question-answering method and device based on vehicle information data. The method comprises the following steps: acquiring a knowledge graph constructed based on the vehicle information data; determining a node semantic coding vector of each node according to the knowledge graph; inputting the adjacent matrix of the knowledge graph and the semantic coding vector of each node into a pre-constructed graph convolutional neural network model to obtain an optimized semantic coding vector corresponding to each node; and according to the received user question and the optimized semantic coding vector corresponding to each node, based on a preset generative large model, obtaining a user question and answer result. By constructing the knowledge graph and inputting the adjacency matrix and the node semantic coding vector of the knowledge graph into the graph convolutional neural network model, the semantic coding vector is optimized, the information recall accuracy is improved, the information omission and misjudgment are reduced, the recall result is more accurate, and the question and answer result generated by the generative large model is more detailed and accurate.

Description

Technical Field

[0001] The invention relates to a knowledge question-answering method and device based on vehicle information data. Background Art

[0002] Nowadays, family cars are very popular. Car owners need to know some information about the vehicle in detail when using it, including various technical parameters, how to use each function, and maintenance steps. Usually this information is recorded in the vehicle manual. With the popularization of intelligent vehicle systems, more and more vehicle manufacturers have digitized the vehicle manual, allowing car owners to easily access this information through the vehicle system. At present, some advanced products have appeared on the market, allowing users to ask questions directly to the vehicle computer, and the vehicle computer can retrieve relevant information in its electronic manual and present it to the user intuitively. This knowledge question and answer method not only improves the user experience, but also makes the presentation of vehicle description information more friendly. At present, the common practice of such products is to extract keywords in user questions, quickly search in the vehicle information database, and instantly display the matched relevant vehicle information to the user. Summary of the invention

[0003] In order to better realize data recall and improve the accuracy of knowledge question answering, an embodiment of the present invention provides a knowledge question answering method and device based on vehicle information data.

[0004] In a first aspect, an embodiment of the present invention provides a knowledge question answering method based on vehicle information data, which may include:

[0005] Obtain a knowledge graph based on vehicle information data;

[0006] Determine a node semantic encoding vector of each node according to the knowledge graph;

[0007] Input the adjacency matrix of the knowledge graph and the semantic encoding vector of each node into a pre-built graph convolutional neural network model to obtain an optimized semantic encoding vector corresponding to each node;

[0008] According to the received user questions and the optimized semantic coding vector corresponding to each node, based on the preset generative big model, the user question and answer result is obtained.

[0009] In one or some optional implementations of the embodiments of the present application, the pre-built graph convolutional neural network model includes a multi-layer improved GCN network;

[0010] The step of inputting the adjacency matrix of the knowledge graph and the semantic encoding vector of each node into a pre-built graph convolutional neural network model to obtain an optimized semantic encoding vector corresponding to each node includes:

[0011] The adjacency matrix and each node semantic encoding vector are input into the pre-built graph convolutional neural network model, and based on the GCN formula, the optimized semantic encoding vector corresponding to each node is obtained:

[0012]

[0013] Among them, X l represents the input of this layer of the improved GCN network, X (l+1) Represents the output of this layer, which is the input of the next layer. represents the degree matrix with the identity matrix added, represents the adjacency matrix with the identity matrix added, Θ l Represents the learnable parameters of this layer, To act as a mask on the adjacency matrix The learnable parameters on It is the matrix element multiplication operation.

[0014] In one or some optional implementations of the embodiments of the present application, the pre-built graph convolutional neural network model includes a multi-layer improved GCN network;

[0015] The step of inputting the adjacency matrix and each of the node semantic coding vectors into a pre-built graph convolutional neural network model to obtain an optimized semantic coding vector corresponding to each node includes:

[0016] The adjacency matrix and each node semantic encoding vector are input into the pre-built graph convolutional neural network model, and based on the GCN formula, the optimized semantic encoding vector corresponding to each node is obtained:

[0017]

[0018] Among them, X l represents the input of this layer of the improved GCN network, X (l+1) Represents the output of this layer, which is the input of the next layer. represents the degree matrix with the identity matrix added, represents the adjacency matrix with the identity matrix added, Θ l Represents the learnable parameters of this layer, To act as a mask on the adjacency matrix The learnable parameters on is the matrix corresponding element multiplication operation, θ 1 ,θ 2 and θ 3 They represent the learnable weights of each order of the graph structure using the third-order Chebyshev polynomial as the convolution kernel.

[0019] In one or some optional implementations of the embodiments of the present application, the pre-built graph convolutional neural network model includes 3 layers of improved GCN networks and 1 CNN network, wherein the first 2 layers of improved GCN networks are connected to form a densely connected convolutional network structure, and the third layer of improved GCN is connected to the CNN network to form a residual network structure;

[0020] The step of inputting the adjacency matrix and each of the node semantic coding vectors into a pre-built graph convolutional neural network model, and obtaining an optimized semantic coding vector corresponding to each node based on a GCN formula, includes:

[0021] Input the adjacency matrix and each of the node semantic encoding vectors into the densely connected convolutional network structure, aggregate neighbor node information based on the GCN formula, and obtain an updated node semantic encoding vector;

[0022] The updated node semantic coding vector is input into the residual network structure, neighbor node information is aggregated based on the GCN formula, and a dimensionality reduction operation is performed on the updated node semantic coding vector to obtain the optimized semantic coding vector.

[0023] In one or some optional implementations of the embodiments of the present application, the pre-built graph convolutional neural network model is trained in the following manner:

[0024] Obtain user question data training set;

[0025] Performing semantic vector encoding on each user question data training sample in the user question data training set to obtain a user question data encoding vector, and inputting the vector into the initial graph convolutional neural network model for iterative training until the model training reaches a preset training target, thereby obtaining a trained graph convolutional neural network model;

[0026] The trained graph convolutional neural network model is tested using a pre-constructed user question data test set, and the trained graph convolutional neural network model is calibrated according to the obtained test results to obtain the pre-constructed graph convolutional neural network model.

[0027] In one or some optional implementations of the embodiment of the present application, after obtaining the optimized semantic coding vector corresponding to each node, the following further includes:

[0028] Bind the optimized semantic coding vector corresponding to each node with the relevant target data information and store them in the vector database;

[0029] The method of obtaining a user question and answer result based on the received user question and the optimized semantic coding vector corresponding to each node and based on a preset generative large model includes:

[0030] Semantically encode the received user questions, and recall the TopN optimized semantic encoding vectors with the highest similarity and bound related target data information from the vector database according to the obtained semantic encoding vector of the question;

[0031] The data text corresponding to the user question and the optimized semantic coding vector of the TopN is input into the preset generative large model, and combined with the bound related target data information to obtain the user question and answer result.

[0032] In one or some optional implementations of the embodiments of the present application, the user question and the data text corresponding to the optimized semantic encoding vector of the TopN are input into the preset generative large model, and combined with the bound related target data information to obtain the user question and answer result, including:

[0033] According to a preset recall similarity threshold, the optimized semantic coding vectors of the TopN are screened to obtain at least one successfully recalled optimized semantic coding vector;

[0034] Obtaining data text corresponding to each of the successfully recalled optimized semantic encoding vectors;

[0035] Inputting the data text corresponding to the user question and the at least one successfully recalled optimized semantic encoding vector into the preset generative large model to obtain a model response;

[0036] The model reply and the bound related target data information are used as the user question and answer result.

[0037] In a second aspect, an embodiment of the present invention provides a knowledge question-answering device based on vehicle information data, which may include:

[0038] A first acquisition module is used to acquire a knowledge graph constructed based on vehicle information data;

[0039] A semantic encoding module, used to obtain a node semantic encoding vector of each node according to the knowledge graph;

[0040] A semantic coding optimization module, used to input the adjacency matrix of the knowledge graph and the semantic coding vector of each node into a pre-built graph convolutional neural network model to obtain an optimized semantic coding vector corresponding to each node;

[0041] The question-answering module is used to obtain the user question-answering result based on the received user question and the optimized semantic coding vector corresponding to each node and based on the preset generative large model.

[0042] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the knowledge question-and-answer method based on vehicle information data as described above.

[0043] In a fourth aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the knowledge question and answer method based on vehicle information data as described above.

[0044] In a fifth aspect, an embodiment of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory, and when the processor executes the computer program, the knowledge question and answer method based on vehicle information data as described above is implemented.

[0045] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0046] The knowledge question-answering method based on vehicle information data provided by the embodiment of the present invention can deeply mine the multi-dimensional associations between vehicle information data by constructing a knowledge graph, input the adjacency matrix and node semantic coding vector of the knowledge graph into the graph convolutional neural network model, optimize the semantic coding vector, improve the accuracy of information recall, reduce information omissions and misjudgments, make the recall results more accurate, and provide the generative large model with more comprehensive information, so that the generative large model can generate more detailed and accurate question-answering results based on user questions and optimized semantic coding vectors. This method is more natural and smooth than directly presenting the original information, and conforms to the user's interaction habits. In addition, the generative large model can generate personalized answers based on the context and the user's questioning method, which enhances the intelligence of the system and improves user satisfaction and user interaction experience.

[0047] The knowledge question-and-answer method based on vehicle information data provided by the embodiment of the present invention combines a knowledge graph, a graph convolutional neural network and a generative large model to form an intelligent knowledge question-and-answer method that can automatically learn and generate answers, reducing the cost of manual intervention. It has a high level of intelligence and can cope with complex and changing problems and user needs, thereby improving service quality and efficiency. Since this method comprehensively considers the semantic information of the text and the association information between the data, the generated answer contains the specific information required by the user, making the answer more detailed and accurate. Compared with traditional keyword recall and ordinary semantic coding recall, the recall accuracy is greatly improved, and the test effect is significant.

[0048] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0049] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0051] Figure 1 A schematic diagram of a flow chart of a knowledge question-answering method based on vehicle information data provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a knowledge graph provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of a graph convolutional neural network model provided by an embodiment of the present invention;

[0054] Figure 4 A schematic diagram of the process of vehicle information data knowledge question and answer interaction provided by an embodiment of the present invention;

[0055] Figure 5 A schematic diagram of the structure of a knowledge question and answer device based on vehicle information data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0057] The inventors have discovered that the vehicle knowledge question and answer solutions in the prior art still have some defects. Keyword-based retrieval systems are difficult to effectively and accurately recall relevant information in vehicle information data. Although there are currently some methods for vector similarity recall based on text encoding models, these methods are limited to only considering semantic information in the language and fail to fully utilize the structural hierarchical information in the data and the correlation between data. Therefore, the effect of recalling information is still not ideal. Based on this, the inventors have made the present invention after further research and development, providing a knowledge question and answer method and device based on vehicle information data.

[0058] Embodiment 1

[0059] In the first embodiment of the present invention, a knowledge question answering method based on vehicle information data is provided. Figure 1 As shown, the method may include the following steps S101-S104:

[0060] S101: Obtain a knowledge graph constructed based on vehicle information data;

[0061] S102: Determine a node semantic encoding vector of each node according to the knowledge graph;

[0062] S103: Input the adjacency matrix of the knowledge graph and the semantic encoding vector of each node into a pre-built graph convolutional neural network model to obtain an optimized semantic encoding vector corresponding to each node;

[0063] S104: Obtaining a user question and answer result based on the received user question and the optimized semantic coding vector corresponding to each node and based on a preset generative large model.

[0064] The knowledge question-answering method based on vehicle information data provided by the embodiment of the present invention can deeply mine the multi-dimensional associations between vehicle information data by constructing a knowledge graph, input the adjacency matrix and node semantic coding vector of the knowledge graph into the graph convolutional neural network model, optimize the semantic coding vector, improve the accuracy of information recall, reduce information omissions and misjudgments, make the recall results more accurate, and provide the generative large model with more comprehensive information, so that the generative large model can generate more detailed and accurate question-answering results based on user questions and optimized semantic coding vectors. This method is more natural and smooth than directly presenting the original information, and conforms to the user's interaction habits. In addition, the generative large model can generate personalized answers based on the context and the user's questioning method, which enhances the intelligence of the system and improves user satisfaction and user interaction experience.

[0065] The knowledge question-and-answer method based on vehicle information data provided by the embodiment of the present invention combines a knowledge graph, a graph convolutional neural network and a generative large model to form an intelligent knowledge question-and-answer method that can automatically learn and generate answers, reducing the cost of manual intervention. It has a high level of intelligence and can cope with complex and changing problems and user needs, thereby improving service quality and efficiency. Since this method comprehensively considers the semantic information of the text and the association information between the data, the generated answer contains the specific information required by the user, making the answer more detailed and accurate. Compared with traditional keyword recall and ordinary semantic coding recall, the recall accuracy is greatly improved, and the test effect is significant.

[0066] In the embodiment of the present invention, the knowledge graph in the above step S101 includes a plurality of nodes and edges between each two nodes having correlation, wherein the nodes are used to represent the vehicle information item text segments obtained by disassembling the vehicle information data. Among them, two nodes having correlation may be that the data of the vehicle information item text segments corresponding to the two nodes have a hierarchical relationship in the original vehicle information data, or the data of the vehicle information item text segments corresponding to the two nodes have correlation in content.

[0067] When constructing a knowledge graph based on vehicle information data, all vehicle information data, that is, the data recorded in the vehicle manual, can be manually disassembled and split into multiple vehicle information entry text segments. Since the granularity of data disassembly and segmentation will affect the amount of content contained in the vehicle information entry text segment, in order to ensure that the information of the recalled data is more accurate during the subsequent recall, the split granularity is as detailed as possible, and each disassembled vehicle information entry text segment is used as a node in the knowledge graph. After that, the edges in the graph are constructed based on the original hierarchical relationship of these data in the vehicle manual. In this way, the original vehicle information data is converted into the form of a knowledge graph, which contains the original hierarchical relationship of these vehicle information.

[0068] Furthermore, for nodes without hierarchical association, it is also possible to manually determine whether the text segments of the vehicle information items corresponding to the nodes are relevant in content. For nodes that have no hierarchical relationship in the original vehicle manual but are related to each other in content, edges between these nodes are constructed on the graph to obtain the above-mentioned knowledge graph constructed based on vehicle information data.

[0069] For example, refer to Figure 2 As shown in Figure 1, the vehicle manual is disassembled and segmented to obtain 9 vehicle information item text segments as nodes of the knowledge graph. Then, 8 edges in the graph are constructed based on the original hierarchical relationship of these 9 data in the vehicle manual (such as Figure 2 Furthermore, for the two nodes “Safety Instructions” and “Driving Safety” which have no hierarchical relationship in the original vehicle manual but are related to each other in content, corresponding edges are constructed on the graph (such as Figure 2 The red line in the figure shows the final knowledge graph based on vehicle information data.

[0070] In an optional embodiment, in the above step S102, the node semantic coding vector of each node is determined according to the knowledge graph. Specifically, each node in the knowledge graph is semantically encoded to obtain the node semantic coding vector of each node.

[0071] In the embodiment of the present invention, each node in the knowledge graph corresponds to a vehicle information entry text segment. The vehicle information entry text segment corresponding to each node is semantically encoded using a pre-trained text encoding model, and these data are converted into a node semantic encoding vector containing semantic information. The vehicle information entry text segment corresponding to each node on the knowledge graph can be represented by a node semantic encoding vector containing semantic information.

[0072] In a specific embodiment, the text encoding model obtained by the above pre-training can be a Chinese encoding model using a BERT-like model. Of course, in some other embodiments, those skilled in the art can also select other suitable text encoding models to implement the semantic vector encoding of the node.

[0073] In an embodiment of the present invention, the adjacency matrix of the knowledge graph described in the above step S103 is a two-dimensional matrix, which is used to represent the relationship between nodes in the knowledge graph.

[0074] In an optional embodiment, the pre-built graph convolutional neural network model includes a multi-layer improved GCN network (Graph Convolutional Networks, graph convolutional neural network);

[0075] Correspondingly, in the above step S103, the adjacency matrix of the knowledge graph and the semantic encoding vector of each node are input into a pre-built graph convolutional neural network model to obtain the optimized semantic encoding vector corresponding to each node, which specifically includes:

[0076] The adjacency matrix and each node semantic encoding vector are input into the pre-built graph convolutional neural network model, and based on GCN formula 1, the optimized semantic encoding vector corresponding to each node is obtained:

[0077]

[0078] Among them, X l represents the input of this layer of the improved GCN network, X (l+1) Represents the output of this layer, which is the input of the next layer. represents the degree matrix with the identity matrix added, represents the adjacency matrix with the identity matrix added, Θ l Represents the learnable parameters of this layer, To act as a mask on the adjacency matrix The learnable parameters on It is the matrix element multiplication operation.

[0079] In an embodiment of the present invention, a graph convolutional neural network model is used to adjust and optimize the vector value of each node in the knowledge graph, so that the semantic coding vector of each node further includes graph structure information on the basis of semantic information, thereby obtaining an optimized semantic coding vector.

[0080] In order to better mine the structural hierarchical information in the data and the correlation between the data, the inventors considered introducing a graph convolutional neural network to further explore the data features. Since a convolutional neural network is to be built on the graph structure, GCN can be used as the convolution layer. In the existing graph convolutional neural network model, the existing basic formula of GCN is as follows:

[0081]

[0082] Where X l Represents the input of this layer, X (l+1) Represents the output of this layer, which is the input of the next layer. represents the degree matrix with the identity matrix added, represents the adjacency matrix with the identity matrix added, Θ l Represents the learnable parameters of this layer.

[0083] The inventors found that since the knowledge graph constructed based on the vehicle information data in the above step S101 is an undirected graph, and all edges are of the same type and no distinction is made, the above adjacency matrix It can only indicate whether there is an edge between nodes, but cannot mark the importance of the edges between adjacent nodes, and cannot fully capture the difference in importance of different edges in the graph when transmitting information. In this way, when a node aggregates the information of neighboring nodes, it is not conducive to distinguishing these nodes, and it is difficult to define the importance of each edge manually. Based on this, the inventor improved the GCN formula in the prior art and proposed the above GCN formula 1, which realizes the aggregation of information between any node and its neighboring nodes through the above GCN formula 1.

[0084] In the embodiment of the present invention, in the above GCN formula 1, a learnable parameter is added for the importance of the edge By introducing new learnable parameters Can be used directly as a mask on the adjacency matrix The learnable parameters is an adjacency matrix Matrices of the same size, the learnable parameters The size of each element in is not fixed, but can be dynamically obtained through the learning process of the neural network. It allows GCN to train the neural network through a pre-prepared data set. During the training process, the network will continuously adjust the value of this parameter matrix, so that the GCN network can learn the importance of each edge by itself. By introducing this learnable parameter This enables the GCN network to automatically learn the weights of each edge in the adjacency matrix and assign a learnable weight to each edge, thereby better mining the structural hierarchical information in the data and the correlation between the data, enhancing the GCN network's ability to understand the graph data structure and information flow. The optimized semantic coding vector of the output node further includes graph structure information on the basis of semantic information, which significantly improves the accuracy and comprehensiveness of the optimized node semantic coding vector.

[0085] In the embodiment of the present invention, the learnable parameter matrix The initial value of can be random or set based on empirical knowledge.

[0086] In another optional embodiment, the inventor considers that since the above-mentioned GCN network that uses formula 1 to aggregate the information of any node and neighbor nodes only aggregates the features of the first-order neighbors in each convolution process, and the information contained in the high-order neighbor nodes in the vehicle information data map is also useful, the inventor proposes to comprehensively consider the features of the first-order neighbor nodes as the main and the features of the high-order neighbor nodes as the auxiliary. Since from the mathematical essence, the above-mentioned GCN network is a convolution operation performed on the graph structure with 0th and 1st order Chebyshev polynomials as convolution kernels, in order to comprehensively consider the features of the first-order neighbor nodes and the high-order neighbor nodes, the inventor proposes to expand the Chebyshev polynomial to the third order, where the Chebyshev polynomial is as follows:

[0087] T k (Δ)=2ΔT k-1 (Δ)-T k-2 (Δ);

[0088] T 0 (Δ) = 1;

[0089] T 1 (Δ) = Δ;

[0090] In this way, during each convolution, each node will simultaneously obtain information about itself and its first-order, second-order, and third-order neighbor nodes. When extracting features of each order, the 0th-order node, that is, the node features itself, is considered, and the importance of each order except the 0th order is introduced into a learnable parameter to be learned by the network during training. The formula of each layer of the improved GCN network is shown in GCN Formula 2:

[0091]

[0092] Among them, X l represents the input of this layer of the improved GCN network, X (l+1) Represents the output of this layer, which is the input of the next layer. represents the degree matrix with the identity matrix added, represents the adjacency matrix with the identity matrix added, Θ l Represents the learnable parameters of this layer, To act as a mask on the adjacency matrix The learnable parameters on is the matrix corresponding element multiplication operation, θ 1 ,θ 2 and θ 3 They represent the learnable weights of each order of the graph structure using the third-order Chebyshev polynomial as the convolution kernel.

[0093] Correspondingly, in the above step S103, the adjacency matrix and each of the node semantic coding vectors are input into a pre-built graph convolutional neural network model to obtain an optimized semantic coding vector corresponding to each node, which may specifically include:

[0094] The adjacency matrix and each of the node semantic coding vectors are input into a pre-built graph convolutional neural network model, and based on GCN formula 2, an optimized semantic coding vector corresponding to each node is obtained.

[0095] In the embodiment of the present invention, based on the above GCN formula 2, a convolution operation is performed on the graph structure using a third-order Chebyshev polynomial as the convolution kernel, by assigning a learnable parameter θ to each of the first-order, second-order, and third-order neighbor nodes. 1 ,θ 2 and θ 3 , which enables the GCN network to automatically learn and fuse information from neighbors of different orders, and more flexibly handle the complex structures and relationships in graph data, thereby more accurately capturing the correlation between nodes and further improving the accuracy and comprehensiveness of the output optimized semantic encoding vector.

[0096] In a specific embodiment, the pre-built graph convolutional neural network model includes 3 layers of improved GCN networks and 1 CNN network, wherein the first 2 layers of improved GCN networks are connected to form a densely connected convolutional network densenet structure, and the third layer of improved GCN is connected to the CNN network to form a residual network structure.

[0097] Correspondingly, the adjacency matrix and each of the node semantic coding vectors are input into a pre-built graph convolutional neural network model, and based on the GCN formula (i.e., GCN formula 1 or GCN formula 2), an optimized semantic coding vector corresponding to each node is obtained, which may specifically include:

[0098] Input the adjacency matrix and each of the node semantic encoding vectors into the densely connected convolutional network structure, aggregate neighbor node information based on GCN formula 1 or GCN formula 2, and obtain an updated node semantic encoding vector;

[0099] The updated node semantic coding vector is input into the residual network structure, neighbor node information is aggregated based on GCN formula 1 or GCN formula 2, and a dimensionality reduction operation is performed on the updated node semantic coding vector to obtain the optimized semantic coding vector.

[0100] In the embodiment of the present invention, based on the improved GCN network, a network model can be constructed to build a multi-layer graph convolutional neural network model. Figure 3 As shown in the figure, the first two layers of improved GCN networks in the above pre-built graph convolutional neural network model are connected to form a densenet structure, which is mainly used to aggregate neighbor node information. The third layer of improved GCN is connected to the CNN network to form a residual network structure, which is used to reduce the dimension of the vector while aggregating neighbor nodes, facilitating the back propagation of the network.

[0101] For example, Figure 3 The GCN network shown is a network using the GCN formula shown in Formula 2. Each GCN layer contains a feature extraction process of a third-order neighbor, where the numbers in brackets represent the vector dimensions of the input and output, respectively. The concat operation is a vector concatenation operation. Figure 3 The graph convolutional neural network model shown in the figure has a model input that is the semantic encoding vector of each node in the graph and the adjacency matrix that represents the graph structure (i.e., the graph adjacency matrix in the figure), and an output that is the optimized semantic encoding vector of each node after optimization (i.e., the updated vectors of each node in the graph).

[0102] Of course, in some other embodiments, if the GCN network is a graph convolutional network using the GCN formula shown in Formula 1, each GCN layer only includes the feature extraction process of the first-order neighbors. Those skilled in the art can construct a graph convolutional neural network model based on the GCN network. The specific implementation process can refer to Figure 3 The detailed description of the graph convolutional neural network model shown is not repeated here.

[0103] In a specific embodiment, the pre-built graph convolutional neural network model can be trained in the following manner:

[0104] Obtain user question data training set;

[0105] Performing semantic vector encoding on each user question data training sample in the user question data training set to obtain a user question data encoding vector, and inputting the vector into the initial graph convolutional neural network model for iterative training until the model training reaches a preset training target, thereby obtaining a trained graph convolutional neural network model;

[0106] The trained graph convolutional neural network model is tested using a pre-constructed user question data test set, and the trained graph convolutional neural network model is calibrated according to the obtained test results to obtain the pre-constructed graph convolutional neural network model.

[0107] In an embodiment of the present invention, each piece of data in the above-mentioned user question data training set and user question data test set is obtained by collecting user query sentence samples and manually annotating useful or useless labels. Specifically, for each user question, all nodes in the knowledge graph are manually annotated, and information related to answering user questions is annotated as positive samples, and information irrelevant to answering user questions is annotated as negative samples. The annotated sample set can be divided into a training set and a test set according to a certain ratio. For example, 70% of them are used as training sets and 30% as test sets.

[0108] In an embodiment of the present invention, the preset training goal may be that the number of iterations reaches a set number or the model learning rate reaches a preset value. Of course, it may also be other training goals, such as accuracy, recall rate or F1 score.

[0109] Exemplarily, the graph convolutional neural network model can be trained using contrastive learning task objectives. For each user question input, the user question is first encoded into a question vector using the same semantic encoding model as that used to encode the semantic vector of each node in the knowledge graph. Then the goal is set to make the positive sample vector as close to the input question vector as possible and the negative sample vector as far away from the input question vector as possible. A preset iteration threshold is set and the model is iteratively trained. After experiments, the number of training iterations is set to 60 times, and the model can converge to the best effect. The learning rate is 0.02, and the learning rate is reduced to 0.1 times the original every 20 iterations.

[0110] Next, the trained graph convolutional neural network model is tested using a test set. During the test, for each input question vector, the top K data (for example, the first 20 data) with the highest similarity to the input question vector among all nodes are first selected, and then a similarity threshold is set. Nodes above this threshold are considered to be successfully recalled. After experiments, when the similarity threshold is set to 0.84, the highest average accuracy of 0.94 can be obtained on the test set. In comparison, the same test set was used to test the traditional keyword recall and ordinary semantic encoding recall methods, and the average accuracy rates obtained were 0.68 and 0.87, respectively. It can be seen that the graph convolutional neural network model of this scheme helps to improve the accuracy of recall information compared to the existing technology.

[0111] In the embodiment of the present invention, when calculating the similarity between the input question vector and the semantic encoding vector of the node, a similarity calculation method in the prior art may be used, such as cosine similarity or Euclidean distance.

[0112] In an optional embodiment, after obtaining the optimized semantic coding vector corresponding to each node based on the above step S103, the method may further include the following steps:

[0113] The optimized semantic coding vector corresponding to each node is bound to the relevant target data information and stored in the vector database.

[0114] In an embodiment of the present invention, the above-mentioned relevant target data information may be some non-text data such as relevant pictures and videos corresponding to the text segment of the vehicle information entry of the node, and these relevant target data information may be selected artificially based on experience. After obtaining the optimized semantic coding vectors corresponding to all nodes through the above-mentioned step S103, the optimized semantic coding vector corresponding to each node is bound to the relevant target information and falls into the vector database. The vector database has the ability to store vectors on the basis of ordinary databases, as well as the ability to quickly calculate and recall similar vectors based on input vectors.

[0115] Accordingly, in the above step S104, according to the received user question and the optimized semantic coding vector corresponding to each node, based on the preset generative big model, a user question and answer result is obtained, which specifically includes:

[0116] Semantically encode the received user questions, and recall the TopN optimized semantic encoding vectors with the highest similarity and bound related target data information from the vector database according to the obtained semantic encoding vector of the question;

[0117] The data text corresponding to the user question and the optimized semantic coding vector of the TopN is input into the preset generative large model, and combined with the bound related target data information to obtain the user question and answer result.

[0118] In an embodiment of the present invention, when semantic encoding is performed on the received user questions, the same semantic encoding model as that used when semantic vector encoding is performed on each node in the knowledge graph can be used.

[0119] In a specific embodiment, the user question and the data text corresponding to the optimized semantic coding vector of the TopN are input into the preset generative large model, and combined with the bound related target data information to obtain the user question and answer result, which may specifically include:

[0120] According to a preset recall similarity threshold, the optimized semantic coding vectors of the TopN are screened to obtain at least one successfully recalled optimized semantic coding vector;

[0121] Obtaining data text corresponding to each of the successfully recalled optimized semantic encoding vectors;

[0122] Inputting the data text corresponding to the user question and the at least one successfully recalled optimized semantic encoding vector into the preset generative large model to obtain a model response;

[0123] The model reply and the bound related target data information are used as the user question and answer result.

[0124] In an embodiment of the present invention, the preset recall similarity threshold may be a similarity threshold determined experimentally when the trained graph convolutional neural network model is tested using the test set mentioned above. Exemplarily, the preset recall similarity threshold is set to 0.84.

[0125] In the embodiment of the present invention, the inventor found that the current form of presentation of relevant vehicle information questions and answers is to directly present the original information of the data on the vehicle manual to the user. Such a form of presentation is more like a retrieval system rather than a question-and-answer system, and lacks a friendly interactive experience for users. In order to improve the user's interactive experience, the inventor proposes that by inputting the data text corresponding to the user's question and the optimized semantic coding vector that has been successfully recalled into a generative large model, the generative large model outputs the model reply, and finally presents the model reply and the data-bound video and / or picture information to the user together, completing a knowledge question-and-answer interaction of the vehicle information data. Such a question-and-answer process is more natural, smooth and personalized, and can not only provide more accurate and rich information, but also ensure that users can quickly find the information they need, thereby improving the user's interactive experience. Provide users with a more intelligent and convenient vehicle information question-and-answer service.

[0126] For example, refer to Figure 4 As shown, for the input user question, firstly, the same semantic encoding model as that used for encoding the semantic vector of each node in the knowledge graph is used for vector encoding (i.e., the Chinese semantic model encoding in the figure) to obtain the question semantic encoding vector;

[0127] Next, the semantic encoding vector of this question is used as input to calculate the similarity in the above vector database, and the top 20 data with the highest similarity are recalled;

[0128] Next, these 20 items are screened according to the preset recall similarity threshold (i.e., the threshold screening in the figure), and the data with similarity higher than this threshold is determined to be successfully recalled, and at least one optimized semantic encoding vector of successful recall is obtained;

[0129] Next, the user's question text is taken as input, and the data text corresponding to the successfully recalled optimized semantic coding vector is given to the preset generative big model as reference material. Based on the reference material, the generative big model organizes the language of the user's question in a targeted manner and outputs the model response (i.e., the generative big model response in the figure).

[0130] Finally, the model response and the relevant target data information bound to the optimized semantic encoding vector of the successful recall are presented to the user as the user question and answer result, completing a question and answer interaction of vehicle information.

[0131] Embodiment 2

[0132] Based on the same inventive concept, the embodiment of the present invention also provides a knowledge question-answering device based on vehicle information data, referring to Figure 5 As shown, the device comprises:

[0133] A first acquisition module 101 is used to acquire a knowledge graph constructed based on vehicle information data;

[0134] A semantic coding module 102, used to obtain a node semantic coding vector of each node according to the knowledge graph;

[0135] The semantic coding optimization module 103 is used to input the adjacency matrix of the knowledge graph and the semantic coding vector of each node into a pre-built graph convolutional neural network model to obtain an optimized semantic coding vector corresponding to each node;

[0136] The question-and-answer module 104 is used to obtain a user question-and-answer result based on the received user question and the optimized semantic coding vector corresponding to each node and based on a preset generative large model.

[0137] Embodiment 3

[0138] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the knowledge question and answer method based on vehicle information data as described in the above-mentioned embodiment 1 is implemented.

[0139] Embodiment 4

[0140] Based on the same inventive concept, an embodiment of the present invention further provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the knowledge question and answer method based on vehicle information data as described in the above-mentioned embodiment 1.

[0141] Embodiment 5

[0142] Based on the same inventive concept, an embodiment of the present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory, and when the processor executes the computer program, it implements the knowledge question and answer method based on vehicle information data as described in the above embodiment 1.

[0143] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0144] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0145] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0146] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0147] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A knowledge question answering method based on vehicle information data, characterized in that: include: Obtain a knowledge graph based on vehicle information data; Determine a node semantic encoding vector of each node according to the knowledge graph; Input the adjacency matrix of the knowledge graph and the semantic encoding vector of each node into a pre-built graph convolutional neural network model to obtain an optimized semantic encoding vector corresponding to each node; According to the received user questions and the optimized semantic coding vector corresponding to each node, based on the preset generative big model, the user question and answer result is obtained.

2. The method according to claim 1, characterized in that The pre-built graph convolutional neural network model includes a multi-layer improved GCN network; The step of inputting the adjacency matrix of the knowledge graph and the semantic encoding vector of each node into a pre-built graph convolutional neural network model to obtain an optimized semantic encoding vector corresponding to each node includes: The adjacency matrix and each node semantic encoding vector are input into the pre-built graph convolutional neural network model, and based on the GCN formula, the optimized semantic encoding vector corresponding to each node is obtained: Among them, X l Represents the input of this layer of the improved GCN network, X (l+1) Represents the output of this layer, which is the input of the next layer. represents the degree matrix with the identity matrix added, represents the adjacency matrix with the identity matrix added, Θ l Represents the learnable parameters of this layer, To act as a mask on the adjacency matrix The learnable parameters on It is the matrix element multiplication operation.

3. The method according to claim 1, characterized in that The pre-built graph convolutional neural network model includes a multi-layer improved GCN network; The step of inputting the adjacency matrix and each of the node semantic coding vectors into a pre-built graph convolutional neural network model to obtain an optimized semantic coding vector corresponding to each node includes: The adjacency matrix and each node semantic encoding vector are input into the pre-built graph convolutional neural network model, and based on the GCN formula, the optimized semantic encoding vector corresponding to each node is obtained: Among them, X l Represents the input of this layer of the improved GCN network, X (l+1) Represents the output of this layer, which is the input of the next layer. represents the degree matrix with the identity matrix added, represents the adjacency matrix with the identity matrix added, Θ l Represents the learnable parameters of this layer, To act as a mask on the adjacency matrix The learnable parameters on is the matrix corresponding element multiplication operation, θ1, θ2 and θ3 respectively represent the learnable weights of each order of the graph structure using the third-order Chebyshev polynomial as the convolution kernel.

4. The method according to claim 2 or 3, characterized in that: The pre-built graph convolutional neural network model includes 3 layers of improved GCN networks and 1 CNN network, wherein the first 2 layers of improved GCN networks are connected to form a densely connected convolutional network structure, and the third layer of improved GCN is connected to the CNN network to form a residual network structure; The step of inputting the adjacency matrix and each of the node semantic coding vectors into a pre-built graph convolutional neural network model, and obtaining an optimized semantic coding vector corresponding to each node based on a GCN formula, includes: Input the adjacency matrix and each of the node semantic encoding vectors into the densely connected convolutional network structure, aggregate neighbor node information based on the GCN formula, and obtain an updated node semantic encoding vector; The updated node semantic coding vector is input into the residual network structure, neighbor node information is aggregated based on the GCN formula, and a dimensionality reduction operation is performed on the updated node semantic coding vector to obtain the optimized semantic coding vector.

5. The method according to claim 4, characterized in that The pre-built graph convolutional neural network model is trained in the following way: Obtain user question data training set; Performing semantic vector encoding on each user question data training sample in the user question data training set to obtain a user question data encoding vector, and inputting the vector into the initial graph convolutional neural network model for iterative training until the model training reaches a preset training target, thereby obtaining a trained graph convolutional neural network model; The trained graph convolutional neural network model is tested using a pre-constructed user question data test set, and the trained graph convolutional neural network model is calibrated according to the obtained test results to obtain the pre-constructed graph convolutional neural network model.

6. The method according to claim 1, characterized in that After obtaining the optimized semantic encoding vector corresponding to each node, it also includes: Bind the optimized semantic coding vector corresponding to each node with the relevant target data information and store them in the vector database; The method of obtaining a user question and answer result based on the received user question and the optimized semantic coding vector corresponding to each node and based on a preset generative large model includes: Semantically encode the received user questions, and recall the TopN optimized semantic encoding vectors with the highest similarity and bound related target data information from the vector database according to the obtained semantic encoding vector of the question; The data text corresponding to the user question and the optimized semantic coding vector of the TopN is input into the preset generative large model, and combined with the bound related target data information to obtain the user question and answer result.

7. The method according to claim 6, characterized in that The user question and the data text corresponding to the optimized semantic coding vector of the TopN are input into the preset generative large model, and combined with the bound related target data information to obtain the user question and answer result, including: According to a preset recall similarity threshold, the optimized semantic coding vectors of the TopN are screened to obtain at least one successfully recalled optimized semantic coding vector; Obtaining data text corresponding to each of the successfully recalled optimized semantic encoding vectors; Inputting the data text corresponding to the user question and the at least one successfully recalled optimized semantic encoding vector into the preset generative large model to obtain a model response; The model reply and the bound related target data information are used as the user question and answer result.

8. A knowledge question-answering device based on vehicle information data, characterized in that: include: A first acquisition module is used to acquire a knowledge graph constructed based on vehicle information data; A semantic encoding module, used to obtain a node semantic encoding vector of each node according to the knowledge graph; A semantic coding optimization module, used to input the adjacency matrix of the knowledge graph and the semantic coding vector of each node into a pre-built graph convolutional neural network model to obtain an optimized semantic coding vector corresponding to each node; The question-answering module is used to obtain the user question-answering result based on the received user question and the optimized semantic coding vector corresponding to each node and based on the preset generative large model.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the knowledge question and answer method based on vehicle information data described in any one of claims 1 to 7 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the knowledge question and answer method based on vehicle information data described in any one of claims 1 to 7 is implemented.

11. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the knowledge question and answer method based on vehicle information data as described in any one of claims 1-7.

Images