Voice interaction method, device and readable storage medium for intelligent products

Abstract

The present disclosure relates to a voice interaction method, device and readable storage medium for intelligent products. The interaction method includes: collecting voice information through a collection module; when determining that the voice information is a query statement, obtaining all preset object texts in the query statement; determining all subgraphs corresponding to the query statement according to all preset object texts and preset path templates; determining candidate subgraphs according to all subgraphs and preset semantic matching models; determining the question and answer results of the query statement according to the candidate subgraphs; and outputting the voice of the question and answer results through a playback module. In this way, the constraint of obtaining core entities through entity linking is eliminated, and the error of entity linking is avoided from being passed to downstream tasks, thereby improving the accuracy of question and answer. In addition, all preset object texts corresponding to the query statement are obtained, and more pattern subgraphs can be mined based on all the preset object texts in the future, which greatly improves the ability of question and answer and the accuracy of question and answer.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to a voice interaction method, device and readable storage medium for an intelligent product. Background Art

[0002] The intelligent question-answering technology of smart products is a common technology in the field of natural language processing (NLP). It combines many subtasks in the field of NLP. Through intelligent question-answering, users' questions can be answered quickly and accurately, bringing users a superior experience and great convenience.

[0003] Question and answer based on knowledge graph is a form of intelligent question and answer. Question and answer based on knowledge graph aims to find the correct answer from the knowledge graph to answer the user's question. Question and answer based on knowledge graph needs to parse human language into a format that can be used for knowledge graph query, so as to find the correct answer from the huge knowledge graph and accurately answer the user's question. For example, when a user asks "Who is Yao Ming's wife", it needs to be parsed into a queryable statement "select DISTINCT? x where {<Yao Ming><wife>? x}" (here taking SPARQL query language and data acquisition protocol as an example), so that the answer can be queried from the knowledge graph.

[0004] In order to improve the effect of knowledge graph question answering, several key technical problems need to be solved in the process of converting natural language into a format that can be used for knowledge graph query. Suppose it is necessary to parse the user's question query to obtain a structured query statement. First, at the entity level, it is necessary to extract the core entities in the query (for example, the entities that are most important for the semantic expression of the sentence) through entity linking. In practical applications, the user's question query is more complex, usually including multiple core entities, or requires multiple steps of reasoning to solve, making the query parsing more complicated, resulting in lower accuracy of question answering. In addition, after extracting the core entity, it is usually faced with the problem of various description methods of entity attributes in the query. For example, the attribute <wife> can be described in various ways, such as "wife", "marriage object", "entering the marriage hall with...", etc. How to correspond these various description methods to the same entity attribute is also a difficult technical problem, which will also affect the accuracy of question answering. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a voice interaction method, device and readable storage medium for an intelligent product.

[0006] According to a first aspect of an embodiment of the present disclosure, a voice interaction method for an intelligent product is provided, comprising:

[0007] Collect voice information through the collection module;

[0008] When it is determined that the voice information is a question sentence, obtaining all preset object texts in the question sentence;

[0009] Determine all subgraphs corresponding to the query statement according to all the preset object texts and the preset path template;

[0010] Determine a candidate subgraph according to all the subgraphs and a preset semantic matching model;

[0011] Determine a question-answer result of the query statement according to the candidate subgraph;

[0012] The voice of the question and answer result is outputted through the playback module.

[0013] Optionally, the determining a candidate subgraph according to all the subgraphs and a preset semantic matching model includes:

[0014] For each of the subgraphs, preprocess the subgraph to obtain a subgraph sentence corresponding to the subgraph, and form a sentence pair with the subgraph sentence and the query sentence;

[0015] Inputting the sentence pair into the semantic matching model to obtain a first matching degree of the sentence pair output by the semantic matching model, wherein the semantic matching model is obtained by training with sample sentence pairs as model input parameters and the first sample matching degrees of the sample sentence pairs as model output parameters;

[0016] Subgraphs corresponding to the K sentence pairs with the highest first matching degrees are determined as candidate subgraphs, where K is greater than 1 and less than or equal to the number of all subgraphs.

[0017] Optionally, determining a question-answer result of the query statement according to the candidate subgraph includes:

[0018] Determine a target subgraph according to the candidate subgraphs and a preset reordering model;

[0019] The question-answer result of the query statement is determined according to the target subgraph.

[0020] Optionally, determining the target subgraph according to the candidate subgraphs and a preset reordering model includes:

[0021] Obtaining relevant feature data of each of the candidate subgraphs;

[0022] For each of the candidate subgraphs, inputting relevant feature data of the candidate subgraph into the re-ranking model to obtain a second matching degree between the candidate subgraph output by the re-ranking model and the query statement, wherein the re-ranking model is obtained by training with sample subgraphs as model input parameters and the second sample matching degrees of the sample subgraphs as model output parameters;

[0023] The candidate subgraph with the second highest matching degree is determined as the target subgraph.

[0024] Optionally, the relevant feature data includes at least one of the following: a first matching degree of the sentence pair where the candidate subgraph is located, a third matching degree between the candidate subgraph and the query statement, first feature data used to characterize whether specified characters of the query statement exist in the candidate subgraph, and second feature data used to characterize whether the answer type of the question word of the query statement is consistent with the answer type of the candidate subgraph.

[0025] Optionally, the third matching degree is determined in the following manner:

[0026] Respectively determine a first character length of the longest matching character between an entity mention of an entity in the candidate subgraph and the query, a second character length of the longest matching character between an attribute and the query, and a third character length of the longest matching character between an attribute value and the query, and sum the first character length, the second character length, and the third character length to obtain a fourth character length L;

[0027] Sum the character lengths of entity mentions, attributes, and attribute values ​​of the entities in the candidate subgraph to obtain a fifth character length M;

[0028] The ratio of the fourth character length L to the fifth character length M is determined as a third matching degree between the candidate subgraph and the query statement.

[0029] Optionally, when determining that the voice information is a question sentence, obtaining all preset object texts in the question sentence includes:

[0030] When it is determined that the voice information is a query sentence, a mentioned word of the query sentence in the mentioned word set corresponding to the knowledge graph is determined as a target mentioned word;

[0031] For each of the target mentioned words, if the target mentioned word is an entity mentioned word, a preset mapping relationship table is queried to obtain all preset object texts of the target mentioned word; if the target mentioned word is a string attribute mentioned word, the target mentioned word is determined as the object text, wherein the mapping relationship table is used to characterize the correspondence between the entity mentioned word and the object text.

[0032] Optionally, the set of mentioned words corresponding to the knowledge graph is determined in the following manner:

[0033] Get all entity and string attribute values ​​in the knowledge graph;

[0034] Obtaining entity mentions from the entity, and obtaining string attribute mentions from the string attribute value;

[0035] The entity mentioned words and the string attribute mentioned words are used to construct a mentioned word set corresponding to the knowledge graph.

[0036] According to a second aspect of an embodiment of the present disclosure, a voice interaction device for an intelligent product is provided, comprising:

[0037] A collection module, configured to collect voice information through a collection module;

[0038] An acquisition module, configured to acquire all preset object texts in the inquiry sentence when determining that the voice information is an inquiry sentence;

[0039] A first determination module is configured to determine all subgraphs corresponding to the query statement according to all the preset object texts and the preset path template;

[0040] A second determination module is configured to determine a candidate subgraph according to all the subgraphs and a preset semantic matching model;

[0041] A third determination module is configured to determine a question-answer result of the query statement according to the candidate subgraph;

[0042] The output module is configured to output the voice of the question and answer result through the playback module.

[0043] According to a third aspect of an embodiment of the present disclosure, a voice interaction device for an intelligent product is provided, including:

[0044] processor;

[0045] a memory for storing processor-executable instructions;

[0046] Wherein, the processor is configured to:

[0047] Collect voice information through the collection module;

[0048] When it is determined that the voice information is a question sentence, obtaining all preset object texts in the question sentence;

[0049] Determine all subgraphs corresponding to the query statement according to all the preset object texts and the preset path template;

[0050] Determine a candidate subgraph according to all the subgraphs and a preset semantic matching model;

[0051] Determine a question-answer result of the query statement according to the candidate subgraph;

[0052] The voice of the question and answer result is outputted through the playback module.

[0053] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the voice interaction method for the smart product provided in the first aspect of the present disclosure are implemented.

[0054] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0055] The above technical solution is adopted to collect voice information through the collection module, and when it is determined that the voice information is a query statement, all preset object texts in the query statement are obtained, and based on all the preset object texts and the preset path template, all subgraphs corresponding to the query statement are determined, and then based on all the subgraphs and the preset semantic matching module, the candidate subgraphs are determined, and the question and answer results of the query statement are determined according to the candidate subgraphs, and finally the voice of the question and answer results is output through the playback module. In this way, the constraint of obtaining core entities through entity linking is eliminated, and the error of entity linking is avoided from being passed to downstream tasks, thereby improving the accuracy of question and answer. In addition, all the preset object texts in the query statement are obtained, and more pattern subgraphs can be mined based on all the preset object texts in the future, which greatly improves the ability of question and answer and the accuracy of question and answer.

[0056] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0058] Figure 1 The present invention is a flowchart of a voice interaction method for a smart product according to an exemplary embodiment.

[0059] Figure 2 It is a flowchart of a method for determining a set of mentioned words corresponding to a knowledge graph according to an exemplary embodiment.

[0060] Figure 3 The present invention is a flowchart showing a method of obtaining all preset object texts in a query sentence according to an exemplary embodiment.

[0061] Figure 4The present invention is a flowchart of determining a candidate subgraph according to an exemplary embodiment.

[0062] Figure 5 The present invention is a block diagram of a voice interaction device for a smart product according to an exemplary embodiment.

[0063] Figure 6 The figure is a block diagram of a voice interaction device of another smart product according to an exemplary embodiment.

[0064] Figure 7 The figure is a block diagram of a voice interaction device of another smart product according to an exemplary embodiment. DETAILED DESCRIPTION

[0065] In related technologies, question and answer based on knowledge graphs are usually performed in the following two ways. The first way is: retrieval-based. In this way, the core entity is first extracted from the query through entity linking technology. Then, starting from the core entity, all subgraphs are found in the knowledge graph. Then, the subgraphs are sorted through the sorting model, and the most likely subgraph is selected as the structured query statement of the query, so as to query the knowledge graph to obtain the answer. For example, the query is "Who is Andy Lau's wife?", and through entity linking, the core entity extracted from the query is <Andy Lau_Famous Artist in Hong Kong, China>. Then, by querying the knowledge graph, the subgraph associated with the entity <Andy Lau_Famous Artist in Hong Kong, China> is obtained. This entity has attributes such as <Wife>, <Height> and <Date of Birth>, so the subgraph includes <Andy Lau_Famous Artist in Hong Kong, China>-<Wife>- <answer>, <Andy Lau, a famous artist from Hong Kong, China> - <Height> - <answer>and <Andy Lau_Famous Artist from Hong Kong, China>-<Birthdate>- <answer>Etc. Finally, these subgraphs are matched with the query and sorted according to the degree of matching. Matching mainly refers to calculating the semantic similarity between the path corresponding to the subgraph and the query. The commonly used method is to vectorize the subgraph and the query, and then calculate the distance between the vectors. For a path, the closer the distance between it and the query, the higher its ranking. After sorting, the path corresponding to the subgraph ranked first is taken as the correct path, and the answer is finally obtained based on the correct path to answer the user's question. However, in this method, the accuracy of question and answer depends entirely on the accuracy of entity links. If the core entity is extracted incorrectly, this error will be passed downstream, resulting in errors in the answer to the final query. In addition, the queries that can be answered by the retrieval-based method are relatively simple, and generally can only answer one-hop or two-hop questions of a core entity. The ability to parse complex queries is relatively lacking.

[0066] The second method is template-based semantic parsing. In this method, the core entities and entity attributes in the query are extracted through predefined templates. There is no unified standard for the representation of templates. Take the following template as an example: ${person}'s wife is => select DISTINCT? x where {${person}<wife>? x}. This template represents a semantic parsing rule. The left side of "=>" in the template represents the template, which is denoted as template P, and the right side of "=>" represents the structured query statement, which is denoted as query statement S. In this template, ${person} represents an entity with the category of person. In the knowledge graph, each entity has an attribute <category>. If the value of the <category> attribute of an entity is person, then this entity belongs to the scope represented by ${person}. The meaning of this parsing rule is that if a query matches template P, then after replacing ${person} in query statement S with a real entity, the parsing result of the query can be obtained. Using this query statement S, you can get the answer in the knowledge graph. However, the template-based semantic parsing method has limited recall, poor generalization ability, and weak recall ability for long-tail queries. In addition, the richness of templates limits the recall ability of the question-answering system, and the collection of templates requires a lot of time and manpower. The queries input by users are ever-changing, and the number of templates is far behind the changes in queries. Therefore, the recall rate of the template-based semantic parsing method is generally not very objective. In addition, the template-based semantic parsing method mainly solves the head query. If the query input by the user changes slightly, the template cannot cover it. For long-tail queries, the forms vary greatly, and the generalization performance of the template is relatively poor, making it difficult to provide parsing capabilities for long-tail queries. These have greatly limited the applicability and accuracy of the template-based semantic parsing method.

[0067] In view of this, the present disclosure provides a voice interaction method, device and readable storage medium for an intelligent product to improve the accuracy of questions and answers.

[0068] The exemplary embodiments will be described in detail below, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0069] Figure 1 FIG. 1 is a flow chart showing a voice interaction method for a smart product according to an exemplary embodiment. Figure 1 As shown, the voice interaction method may include the following steps.

[0070] In step 11, voice information is collected by a collection module.

[0071] It is worth noting that the voice interaction method provided by the present disclosure can be applied to smart products or servers, and the present disclosure does not specifically limit this. When the voice interaction method can be applied to smart products, the smart product is provided with a collection module for collecting voice information, and the collection module can be, for example, a voice collection device. In addition, the voice information can be input by a user or by other electronic devices, and the present disclosure does not specifically limit this.

[0072] In step S12, when it is determined that the voice information is a question sentence, all preset object texts in the question sentence are acquired.

[0073] It is worth noting that in the present disclosure, whether the voice information is a question statement can be determined by detecting the semantics of the voice information, which is not specifically limited in the present disclosure.

[0074] Among them, all the preset object texts obtained may include entity texts and / or string attribute value texts. It is worth noting that in the present disclosure, all the preset object texts in the query statement are obtained, while in the related art, only the core entities in the query statement are obtained based on the entity link. In this way, on the one hand, the present disclosure can get rid of the limitations of entity links, and on the other hand, more information can be obtained, which is convenient for subsequent mining of more pattern subgraphs based on all the preset object texts, greatly improving the question-answering ability. Among them, the specific method of obtaining all the preset object texts will be described in detail below.

[0075] In step S13, all subgraphs corresponding to the query statement are determined according to all preset object texts and preset path templates.

[0076] In the present disclosure, multiple path templates can be pre-set based on experience, and the path templates can include a path template corresponding to an object text and a path template corresponding to multiple object texts. For example, when all the preset object texts obtained in step S12 are one, the path template corresponding to each preset object text can be used to determine all subgraphs corresponding to the query statement to obtain all subgraphs. When all the preset object texts obtained in step S12 are N, the path template corresponding to each preset i object texts can be used to determine all subgraphs corresponding to the query statement. Wherein, N is greater than 2, and the value range of i is [1, N].

[0077] In the present disclosure, the preset path templates include a path template corresponding to an object text and a plurality of path templates corresponding to object texts. In this way, more candidate path situations can be recalled, which expands the capability boundary of question answering, and especially improves the question answering capability of complex query statements.

[0078] In step 14, candidate subgraphs are determined based on all subgraphs and a preset semantic matching model.

[0079] In step 15, the question-answering result of the query statement is determined according to the candidate subgraph.

[0080] In the present disclosure, a candidate subgraph can be obtained from all subgraphs through a machine learning model, and then the question and answer result of the query statement can be further determined from the candidate subgraph. For example, in the present disclosure, the question and answer result of the query statement can be determined from a preset knowledge graph.

[0081] In step 16, the voice of the question and answer result is output through the playback module.

[0082] The above technical solution is adopted to collect voice information through the collection module, and when it is determined that the voice information is a query statement, all preset object texts in the query statement are obtained, and based on all the preset object texts and the preset path template, all subgraphs corresponding to the query statement are determined, and then based on all the subgraphs and the preset semantic matching module, the candidate subgraphs are determined, and the question and answer results of the query statement are determined according to the candidate subgraphs, and finally the voice of the question and answer results is output through the playback module. In this way, the constraint of obtaining core entities through entity linking is eliminated, and the error of entity linking is avoided from being passed to downstream tasks, thereby improving the accuracy of question and answer. In addition, all the preset object texts in the query statement are obtained, and more pattern subgraphs can be mined based on all the preset object texts in the future, which greatly improves the ability of question and answer and the accuracy of question and answer.

[0083] In order to enable those skilled in the art to better understand the voice interaction method of the intelligent product provided by the present disclosure, a complete embodiment is described below.

[0084] Before describing in detail how to obtain all preset object texts in a query sentence, a method for constructing a set of mentioned words corresponding to the knowledge graph is first described.

[0085] Figure 2 FIG. 1 is a flowchart of a method for determining a set of mentioned words corresponding to a knowledge graph according to an exemplary embodiment. Figure 2 As shown, the method for determining the mentioned word set corresponding to the knowledge graph may include steps S21 to S23.

[0086] In step S21, all entity and string attribute values ​​in the knowledge graph are obtained.

[0087] In step S22, entity mentions are obtained from the entity, and string attribute mentions are obtained from the string attribute values.

[0088] In step S23, a set of mentions corresponding to the knowledge graph is constructed using the entity mentions and the string attribute mentions.

[0089] In the present disclosure, from the SPO triple data of the knowledge graph, all entities and string attribute values are collected to construct a set of mentions S corresponding to the knowledge graph. Exemplarily, for an entity, the mention of the entity is added to the set of mentions S, and for a string attribute value, it is directly added to the set of mentions S without any change. Among them, the entity mention is the name representing the entity. The string attribute mention is the string of the attribute value representing an attribute of the entity.

[0090] For example, if the triple data is <Yao Ming - former investor of Shanghai Men's Basketball Team> - <height> - "226cm", then the entity is <Yao Ming - former investor of Shanghai Men's Basketball Team>, and the entity mention of this entity is "Yao Ming". Therefore, the entity mention "Yao Ming" is added to the set of mentions S. The string attribute value is "226cm", and the string attribute mention of this string attribute value is "226cm". Therefore, the string attribute mention "226" is added to the set of mentions S. In this way, for each triple data, the entity mention and the string attribute mention can be obtained in the above manner, and then the set of mentions S can be constructed.

[0091] It should be noted that for the entity mention, a mapping relationship table D from the entity mention to the entity can also be constructed. Continuing with the above example, the entity is <Yao Ming - former investor of Shanghai Men's Basketball Team>, the entity mention is "Yao Ming", and the mapping relationship from this entity mention to the entity is "Yao Ming": <Yao Ming - former investor of Shanghai Men's Basketball Team>. In this way, for each entity mention, the mapping relationship from the entity mention to the entity can be constructed.

[0092] The following explains how to obtain all the preset object texts in the inquiry statement.

[0093] As Figure 3 shown, Figure 1 in step S12, when it is determined that the voice information is an inquiry statement, obtaining all the preset object texts in the inquiry statement may further include step S121 and step S122.

[0094] In step S121, when it is determined that the voice information is an inquiry statement, the mentions of the inquiry statement located within the set of mentions corresponding to the knowledge graph are determined as the target mentions.

[0095] After determining the mentioned word set S corresponding to the knowledge graph in the above manner, the mentioned words of the query sentence within the mentioned word set S are determined as target mentioned words.

[0096] Assuming the query is "Among Yao Ming's teammates, who is 189cm tall?", by matching with the mention word set S, it is found that the entity mention word "Yao Ming" and the string attribute mention word "189cm" in the query are in the mention word set S, then the entity mention word "Yao Ming" and the string attribute mention word "189cm" are determined as target mention words.

[0097] In step S122, for each target mention word, if the target mention word is an entity mention word, the preset mapping relationship table is queried to obtain all preset object texts of the target mention word; if the target mention word is a string attribute mention word, the target mention word is determined as the object text, wherein the mapping relationship table is used to characterize the correspondence between the entity mention word and the entity.

[0098] Continuing with the above example, the target mentioned words determined are "Yao Ming" and "189cm". For the target mentioned word "Yao Ming", the preset mapping relationship table S is queried to obtain all the preset object texts of the target mentioned word "Yao Ming". Assuming that there are two mapping relationships in the mapping relationship table S, "Yao Ming": <Yao Ming_Former Shanghai Men's Basketball Investor>, "Yao Ming": <Yao Ming_Chairman of the Yao Ming Group>, then all the preset object texts of the target mentioned word "Yao Ming" are <Yao Ming_Former Shanghai Men's Basketball Investor> and <Yao Ming_Chairman of the Yao Ming Group>. For the target mentioned word "189cm", since the target mentioned word is a string attribute mentioned word and there is no mapping relationship, the target mentioned word can be directly determined as the object text. In this way, all the preset object texts corresponding to the query statement determined include: <Yao Ming_Former Shanghai Men's Basketball Investor>, <Yao Ming_Chairman of the Yao Ming Group> and "189cm".

[0099] After all preset object texts in the query statement are determined, all subgraphs corresponding to the query statement are determined according to all the preset object texts and the preset path template.

[0100] For example, it is assumed that the preset path templates include a path template corresponding to one object text, a path template corresponding to two object texts, and a path template corresponding to three object texts.

[0101] Table 1 shows the preset path template.

[0102]

[0103]

[0104] Among them, in Table 1, the path template starting from one object text is the path template corresponding to one object text, the path template starting from two object texts is the path template corresponding to the two object texts, and the path template starting from three object texts is the path template corresponding to the three object texts. It is worth noting that Table 1 only shows the path template corresponding to one object text, the path template corresponding to two object texts, and the path template corresponding to three object texts. According to actual needs, path templates corresponding to four object texts, five object texts, or more object texts can also be set, and the present disclosure does not make specific limitations on this.

[0105] For example, for each object text among all the preset object texts obtained in step 12, starting from the object text in the knowledge graph, a subgraph is determined along a path template corresponding to an object text pre-set in Table 1. For example, if the object text is <Yao Ming_Former Shanghai Men's Basketball Investor>, and <Yao Ming__Former Shanghai Men's Basketball Investor> is used as the entity Subject, according to the preset one-hop path template "Subject-Predicate-Answer", the subgraphs that can be determined include <Yao Ming_Former Shanghai Men's Basketball Investor>-<Wife>-Answer, <Yao Ming_Former Shanghai Men's Basketball Investor>-<Height>-Answer, <Yao Ming_Former Shanghai Men's Basketball Investor>-<Birthplace>-Answer, etc.; if <Yao Ming__Former Shanghai Men's Basketball Investor> is used as the attribute value Object, according to the preset one-hop path template "Answer-Predicate-Object", the subgraphs that can be determined include Answer-<Spouse>-<Yao Ming_Former Shanghai Men's Basketball Investor>, Answer-<Teammate>-<Yao Ming_Former Shanghai Men's Basketball Investor>, Answer-<Chairman>-<Yao Ming_Former Shanghai Men's Basketball Investor>, etc. Among them, if the object text is a string attribute value text, the string attribute value text can only be used as an attribute value Object. For the object text "189cm", according to the preset one-hop path template "Answer-Predicate-Object", the subgraphs that can be determined include Answer-<height>-"189cm" and so on.

[0106] For another example, the object text is <Yao Ming_Former Shanghai Men's Basketball Investor>, and <Yao Ming__Former Shanghai Men's Basketball Investor> is used as the entity Subject. According to the preset two-hop path template "Subject-Predicate-Intermediate-Predicate-Answer", the subgraphs that can be determined are <Yao Ming_Former Shanghai Men's Basketball Investor>-<Wife>-Intermediate-<Birthplace>-Answer, etc. The Intermediate node obtained by the first hop query is used as the entity Subject of the second hop; with <Yao Ming__Former Shanghai Men's Basketball Investor> as the attribute value Object, according to the preset two-hop path template "Intermediate-Predicate-Object-Intermediate-Predicate-Answer", The subgraphs determined include Intermediate-<husband>-<Yao Ming_Former Shanghai Men's Basketball Investor>-Intermediate-<birthplace>-Answer, etc. The Intermediate node obtained by the first hop query is used as the entity Subject of the second hop; with <Yao Ming__Former Shanghai Men's Basketball Investor> as the attribute value Object, according to the preset two-hop path template "Intermediate-Predicate-Object-Answer-Predicate-Intermediate", the subgraphs that can be determined include Intermediate-<husband>-<Yao Ming_Former Shanghai Men's Basketball Investor>-Answer-<alumni>-Intermediate, etc., and the Intermediate node obtained by the first hop query is used as the attribute value Object of the second hop. Among them, for the object text "189cm", according to the preset two-hop path template "Intermediate-Predicate-Object-Intermediate-Predicate-Answer", the subgraphs that can be determined include Intermediate-<height>-"189cm"-Answer-<alumni>-Intermediate, etc., and the Intermediate node obtained by the first hop query is used as the attribute value Object of the second hop.

[0107] For example, for any two object texts among all the preset object texts obtained in step 12, starting from the two object texts in the knowledge graph, a subgraph is determined along the path template corresponding to the two object texts preset in Table 1. For example, the object texts are <Andy Lau_Famous Artist in Hong Kong, China> and <Jacky Cheung_Famous Artist in Hong Kong, China>, and <Andy Lau_Famous Artist in Hong Kong, China> and <Jacky Cheung_Famous Artist in Hong Kong, China> are used as entity Subjects or attribute values ​​Object. According to the path template "Two-Entity-Path", the subgraphs that can be determined are <Andy Lau_Famous Artist in Hong Kong, China>-<Starring Movie>-Answer-<Jacky Cheung_Famous Artist in Hong Kong, China>-<Starring Movie>-Answer, <Andy Lau_Famous Artist in Hong Kong, China>-<Starring Movie>-Answer-<Producer>-<Jacky Cheung_Famous Artist in Hong Kong, China>, etc., which represent a subgraph in which two nodes are connected by a common node. In the first subgraph, both nodes are entity Subjects, and in the second subgraph, the two nodes are entity Subjects and attribute values ​​Objects, respectively. Among them, if one of any two object texts is an entity text and the other is a string attribute value text, the string attribute value text can only be used as an attribute value Object. For the object text <Andy Lau_Famous Hong Kong Artist> and "120 minutes", "120 minutes" can only be used as an attribute value Object. With the object text <Andy Lau_Famous Hong Kong Artist> as the entity Subject or attribute value Object, and "120 minutes" as the attribute value Object, according to the path template "Two-Entity-Path", the subgraphs that can be determined are <Andy Lau_Famous Hong Kong Artist>-<Starring Movie>-Answer-<Duration>-"120 minutes", Answer-<Producer>-<Andy Lau_Famous Hong Kong Artist>-Answer-<Duration>-"120 minutes", etc.

[0108] For another example, the object texts are <Andy Lau_Famous Artist of Hong Kong, China> and <Jacky Cheung_Famous Artist of Hong Kong, China>, with <Andy Lau_Famous Artist of Hong Kong, China> and <Jacky Cheung_Famous Artist of Hong Kong, China> as entity Subjects or attribute value Objects, and according to the path template "Two-Entity-Path-1-hop", the subgraphs that can be determined are <Andy Lau_Famous Artist of Hong Kong, China>-<Starring Movies>-Intermediate-<Jacky Cheung_Famous Artist of Hong Kong, China>-<Starring Movies>-Intermediate-Intermediate-<Duration>-Answer, <Andy Lau_Famous Artist of Hong Kong, China>-<Starring Movies>-Intermediate-<Jacky Cheung_Famous Artist of Hong Kong, China>-<Starring Movies>-Intermediate-Answer-<Representative Works>-Intermediate, etc., which means that the intermediate nodes are first obtained through "Two-Entity-Path", and then the answers are obtained through one hop using the intermediate nodes as entity Subjects or attribute value Objects. Among them, for the object text <Andy Lau_Famous Artist of Hong Kong, China> and "120 minutes", with the object text <Andy Lau_Famous Artist of Hong Kong, China> as the entity Subject or attribute value Object, and "120 minutes" as the attribute value Object, according to the path template "Two-Entity-Path-1-hop", the subgraphs that can be determined include Intermediate-<Producer>-<Andy Lau_Famous Artist of Hong Kong, China>-Intermediate-<Duration>-"120 minutes"-Intermediate-<Director>-Answer, Intermediate-<Producer>-<Andy Lau_Famous Artist of Hong Kong, China>-Intermediate-<Duration>-"120 minutes"-Answer-<Representative Works>-Intermediate, etc., which means that the intermediate node is first obtained through "Two-Entity-Path", and then the intermediate node is used as the entity Subject or attribute value Object, and the answer is obtained through one hop.

[0109] As another example, for any three object texts among all the preset object texts obtained in step 12, starting from these three object texts in the knowledge graph, a subgraph is determined along the path template corresponding to the three object texts preset in Table 1. For example, the object texts are <Andy Lau_Famous Artist in Hong Kong, China>, <Jacky Cheung_Famous Artist in Hong Kong, China>, and <Aaron Kwok_Famous Artist in Hong Kong, China>, and <Andy Lau_Famous Artist in Hong Kong, China>, <Jacky Cheung_Famous Artist in Hong Kong, China>, and <Aaron Kwok_Famous Artist in Hong Kong, China> are used as entity Subjects or attribute values ​​Object. According to the path template "Three-Entity-Path", the subgraphs that can be determined are <Andy Lau_Famous Artist in Hong Kong, China>-<Starring Movies>-Answer-<Jacky Cheung_Famous Artist in Hong Kong, China>-<Starring Movies>-Answer-<Jacky Cheung_Famous Artist in Hong Kong, China>-<Starring Movies>-Answer, etc., which represent a subgraph in which three nodes are connected by a common node. Among them, if there is a string attribute value text in the three object texts, the string attribute value text can only be used as an attribute value Object. The three object texts are <Andy Lau_Famous Hong Kong Artist>, <Jacky Cheung_Famous Hong Kong Artist> and "120 minutes". With <Andy Lau_Famous Hong Kong Artist> and <Jacky Cheung_Famous Hong Kong Artist> as entity Subject or attribute value Object, and "120 minutes" as attribute value Object, according to the path template "Three-Entity-Path", the subgraphs that can be determined are <Andy Lau_Famous Hong Kong Artist>-<Starring Movie>-Answer-<Jacky Cheung_Famous Hong Kong Artist>-<Starring Movie>-Answer-Answer-<Duration>- "120 minutes", etc., which represents a subgraph with three nodes connected by a common node.

[0110] For another example, the object texts are <Andy Lau_Famous Artist of Hong Kong, China>, <Jacky Cheung_Famous Artist of Hong Kong, China>, and <Aaron Kwok_Famous Artist of Hong Kong, China>, and <Andy Lau_Famous Artist of Hong Kong, China>, <Jacky Cheung_Famous Artist of Hong Kong, China>, and <Aaron Kwok_Famous Artist of Hong Kong, China> are used as entity Subjects or attribute value Objects. According to the path template "Three-Entity-Path-1-hop", the subgraphs that can be determined are <Andy Lau_Famous Artist of Hong Kong, China>-<Starring Movies>-Intermediate-<Jacky Cheung_Famous Artist of Hong Kong, China>-<Starring Movies>-Intermediate-<Aaron Kwok_Famous Artist of Hong Kong, China>-<Starring Movies>-Intermediate-Intermediate-<Duration>-Answer, etc., which means that the intermediate nodes are first obtained through "Three-Entity-Path", and then the intermediate nodes are used as entity Subjects or attribute value Objects, and the answers are obtained through one hop. Among them, for the object texts <Andy Lau_Famous Artist of Hong Kong, China>, <Jacky Cheung_Famous Artist of Hong Kong, China>, and "120 minutes", <Andy Lau_Famous Artist of Hong Kong, China>, <Jacky Cheung_Famous Artist of Hong Kong, China> are used as entity Subjects or attribute value Objects, and "120 minutes" is used as attribute value Object. According to the path template "Three-Entity-Path-1-hop", the subgraphs that can be determined are <Andy Lau_Famous Artist of Hong Kong, China>-<Starring Movie>-Intermediate-<Jacky Cheung_Famous Artist of Hong Kong, China>-<Starring Movie>-Intermediate-<Duration>-"120 minutes"-Intermediate-<Production Company>-Answer, etc., which means that the intermediate nodes are first obtained through "Three-Entity-Path", and then the intermediate nodes are used as entity Subjects or attribute value Objects, and the answers are obtained through one hop respectively.

[0111] According to the above method, all subgraphs corresponding to the query statement can be determined based on all the preset object texts and preset path templates. In this way, through subgraph mining based on path templates, more candidate path situations can be recalled, expanding the ability boundary of question answering, especially improving the question answering ability of complex query statements.

[0112] For example, Figure 4 FIG. 1 is a flowchart showing a method of determining a candidate subgraph according to an exemplary embodiment. Figure 4 As shown, Figure 1 Step S14 in the method may include steps S141 to S143.

[0113] In step S141, for each subgraph, the subgraph is preprocessed to obtain a subgraph sentence corresponding to the subgraph, and the subgraph sentence is combined with a query sentence to form a sentence pair.

[0114] For example, the subgraph can be processed as follows: remove the Intermediate node in the subgraph and replace the Answer node with a specified symbol. It is worth noting that if the Intermediate node does not exist, there is no need to remove the Intermediate node in the subgraph. In addition, the specified symbol that replaces the Answer node cannot be a symbol that has appeared in the query statement, that is, the specified symbol used to replace the Answer node can only represent the meaning of the Answer node. For example, the symbol "^" has never appeared in the query statement, so the "^" symbol can be used to replace the Answer node.

[0115] For example, suppose the query is "When was Yao Ming's wife born?", and the corresponding subgraph is <Yao Ming_Former Shanghai Men's Basketball Investor>-<Wife>-Intermediate-<Birthplace>-Answer. Remove the Intermediate node and replace the Answer node with the "^" symbol, and the subgraph statement is "Yao Ming former Shanghai Men's Basketball Investor wife birthplace^". In this way, "When was Yao Ming's wife born" and "Yao Ming former Shanghai Men's Basketball Investor wife birthplace^" form a sentence pair.

[0116] In step 142, the sentence pair is input into a semantic matching model to obtain a first matching degree of the sentence pair output by the semantic matching model.

[0117] The semantic matching model can be trained through the following steps:

[0118] Step a, prepare positive samples. Prepare 5000 queries, and manually parse each query to get the correct subgraph corresponding to the query (hereinafter referred to as the parsed subgraph), and perform the following operations on the parsed subgraph: remove the Intermediate node, replace the Answer node with the "^" symbol to get the subgraph statement, then combine the subgraph statement with the query statement to form a sentence pair, which is recorded as a positive sample, and the first sample matching degree of the manually labeled positive sample is 100%.

[0119] Step b, prepare negative samples. For each of the 5000 queries in step a, refer to the above-mentioned method of determining all subgraphs corresponding to the query statement, determine all subgraphs corresponding to the query, and remove the parsed subgraph from all the determined subgraphs. Then, perform the following operations on the subgraph after removing the parsed subgraph: remove the Intermediate node, and replace the Answer node with the "^" symbol to obtain the subgraph statement. After that, the subgraph statement and the query statement are combined into a sentence pair. In this way, each query can be composed of multiple sentence pairs. Among the sentence pairs composed of 5000 queries, 50,000 sentence pairs are randomly selected as negative samples, and the first sample matching degree of each negative sample is manually marked as 0.

[0120] Step c, train the BERT model with the text pair classification task based on the positive and negative samples received in step a and step b. During training, set the total number of characters in each sample to no more than 64 (i.e., max_seq_length=64), the number of samples output each time to 32 (i.e., train_batch_size=32), the learning rate of the model to 2e-5 (i.e., learning_rate=2e-5), and the number of training rounds to 2 (i.e., num_train_epochs=2). That is, when the model training rounds reach 2 rounds, the training ends, and the model obtained at this time is the semantic matching model.

[0121] After the semantic matching model is trained, the candidate subgraphs are determined from all subgraphs using the semantic matching model.

[0122] For example, the sentence pair can be input into the semantic matching model to obtain the first matching degree of the sentence pair output by the semantic matching model, wherein the first matching degree is used to characterize the matching degree between the subgraph sentence in the sentence pair and the query sentence. For each sentence pair, the sentence pair can be input into the semantic matching model to obtain the first matching degree of the sentence pair. In this way, the first matching degree of each sentence pair can be obtained.

[0123] In step 143, subgraphs corresponding to K sentence pairs with the highest first matching degree are determined as candidate subgraphs, where K is greater than 1 and less than or equal to the number of all subgraphs.

[0124] It is worth noting that, assuming that the number of all subgraphs determined in step 13 is n, correspondingly, n sentence pairs and the first matching degrees of the n sentence pairs can be obtained, that is, n first matches are obtained, and then the subgraphs corresponding to the K sentence pairs with the highest first matching degrees are determined from the n first matching degrees, and the subgraphs corresponding to the determined K sentence pairs are determined as candidate subgraphs. Wherein, K is an integer greater than 1 and less than or equal to n.

[0125] For example, K is 10. After obtaining the first matching degree of each sentence pair, the sentence pairs are sorted according to the first matching degree, and the 10 subgraphs corresponding to the first 10 sentence pairs are determined as candidate subgraphs. Then, the question and answer result of the query statement is determined according to the candidate subgraphs.

[0126] For example, a target subgraph may be determined based on the candidate subgraphs and a preset reordering model, and then a question-answering result of the query statement may be determined based on the target subgraph.

[0127] In this embodiment, a candidate subgraph is first determined based on a semantic matching model, and then a target subgraph is further determined based on a candidate subgraph reordering model. Compared with a solution of directly determining the target subgraph based on a semantic matching model, the accuracy of determining the target subgraph is improved, and the question and answer results of the query statement can be more accurately queried based on the accurate target subgraph.

[0128] The specific implementation method of determining the target subgraph according to the candidate subgraphs and the preset reordering model may include steps (1) to (3):

[0129] Step (1), obtaining relevant feature data of each candidate subgraph. The relevant feature data may include at least one of the following: a first matching degree of the sentence pair in which the candidate subgraph is located, a third matching degree between the candidate subgraph and the query sentence, first feature data used to characterize whether a specified character of the query sentence exists in the candidate subgraph, and second feature data used to characterize whether the answer type of the question word in the query sentence is consistent with the answer type of the candidate subgraph.

[0130] It is worth noting that when determining the second feature data, it is first necessary to extract the interrogative word of the query statement, and further determine the answer type of the interrogative word based on the interrogative word. For example, if the interrogative word in the query statement is "who" or "which", the answer type of the interrogative word is a person type. Alternatively, if the interrogative word in the query statement is "where" or "where", the answer type of the interrogative word is a location type. Alternatively, if the interrogative word in the query statement is "when" or "which day", the answer type of the interrogative word is a date type. Alternatively, if the interrogative word in the query statement is "how big" or "how much", the answer type of the interrogative word is a number type. Next, determine the answer type of the candidate subgraph. For example, the answer corresponding to the candidate subgraph can be obtained by querying the knowledge graph according to the candidate subgraph, and then the answer type is determined according to the queried answer. After determining the answer type of the interrogative word of the query statement and the answer type of the candidate subgraph in the above manner, determine whether the two are consistent. If they are consistent, the second feature data is 1, otherwise the second feature data is 0.

[0131] In the present disclosure, the third matching degree may be determined in the following manner:

[0132] First, determine the first character length of the longest matching character between the entity mention of the entity and the query statement, the second character length of the longest matching character between the attribute and the query statement, and the third character length of the longest matching character between the attribute value and the query statement, and sum the first character length, the second character length, and the third character length to obtain a fourth character length L. For example, the query statement is "Who is Yao Ming's wife?", and the candidate subgraph is <Yao Ming_Former Shanghai Men's Basketball Investor>-<Wife>-Answer, then the entity Subject is "Yao Ming_Former Shanghai Men's Basketball Investor", the entity mention of the entity is "Yao Ming", and the attribute Predicate is "Wife". The attribute value Object is a specified symbol, then the first character length of the longest matching character between the entity mention and the query statement is 2, the second character length of the longest matching character between the attribute Predicate and the query statement is 0, and the third character length of the longest matching character between the attribute value Object and the query statement is 0, then the fourth character length L obtained by summing the first character length, the second character length, and the third character length is 2. It is worth noting that the character length mentioned in the present disclosure may be the number of characters.

[0133] Next, the character lengths of the entity mentions, attributes, and attribute values ​​of the entities in the candidate subgraph are summed to obtain the fifth character length M. Using the above example, the entity mention is "Yao Ming" and the attribute Predicate is "wife", then the character lengths of the entities, attributes, and attribute values ​​in the candidate subgraph are summed to obtain the fifth character length M, which is 4.

[0134] Finally, the ratio R of the fourth character length L to the fifth character length M is determined as the third matching degree between the candidate subgraph and the query statement.

[0135] In addition, the first feature data can be determined in the following manner: the designated character is a time or numeric string. For example, there is a time string (such as "October 2020") or a numeric string (such as "21" in "The 21st World Cup") in the query statement. If the time string or numeric string in the query statement appears in the candidate subgraph, the first feature data can be identified as 1, otherwise it is identified as 0. Alternatively, the designated character is a special symbol. For example, there are feature symbols (such as "", (), "", '', etc.) in the query statement. If the feature symbol in the query statement has appeared in the candidate subgraph, the first feature data can be 1, otherwise it is identified as 0.

[0136] Step (2): for each candidate subgraph, input relevant feature data of the candidate subgraph into the re-ranking model to obtain a second matching degree between the candidate subgraph output by the re-ranking model and the query statement, wherein the second matching degree is used to characterize the matching degree between the candidate subgraph and the query statement.

[0137] In the present disclosure, the reordering model can be trained by the following steps:

[0138] Step d, prepare samples. First, prepare 100 queries, and for each query, manually parse the correct subgraph corresponding to the query (hereinafter referred to as the parsed subgraph), and for each query, determine the candidate subgraph set in the above manner, and the candidate subgraph set includes the determined multiple candidate subgraphs. Determine whether the parsed subgraph is located in the candidate subgraph set. If not, add the parsed subgraph to the candidate subgraph set as a candidate subgraph. After that, take the parsed subgraph in the candidate subgraph set as a positive sample, and take the other candidate subgraphs in the candidate subgraph set except the parsed subgraph as negative samples. Moreover, the candidate subgraph corresponding to the positive sample has a 100% match with the second sample of the query statement, and the candidate subgraph corresponding to the negative sample has a 0 match with the second sample of the query statement.

[0139] Step e: determining the relevant feature data of each sample. The relevant feature data of each sample can be determined by referring to the above-mentioned method of obtaining the relevant feature data of the candidate sub-graph. No further details will be given here.

[0140] In step f, the relevant feature data of each sample is used as the model input parameter, and the matching degree between the candidate subgraph corresponding to the sample and the second sample of the query sentence is used as the model output parameter to train the random forest model. When the training is completed, a re-ranking model is obtained.

[0141] In this way, the second matching degree between each candidate subgraph and the query sentence can be obtained by using the trained re-ranking model.

[0142] Step (3), determining the candidate subgraph with the second highest matching degree as the target subgraph.

[0143] After obtaining the target subgraph according to any of the above embodiments, the question and answer result of the query statement is determined according to the target subgraph. For example, the question and answer result of the query statement can be queried from the knowledge graph according to the target subgraph.

[0144] By adopting the above technical solution, the target subgraph can be accurately obtained through the semantic matching model and the re-ranking model in turn. Then, based on the accurate target subgraph, more accurate question and answer results can be obtained by querying in the knowledge graph, further improving the accuracy of question and answer.

[0145] Figure 5 is a block diagram of a voice interaction device for a smart product according to an exemplary embodiment. Figure 5 , the voice interaction device 500 based on the smart product may include:

[0146] The collection module 501 is configured to collect voice information through a collection module;

[0147] The acquisition module 502 is configured to acquire all preset object texts in the inquiry sentence when determining that the voice information is an inquiry sentence;

[0148] A first determination module 503 is configured to determine all subgraphs corresponding to the query statement according to all the preset object texts and the preset path template;

[0149] A second determination module 504 is configured to determine a candidate subgraph according to all the subgraphs and a preset semantic matching model;

[0150] A third determination module 505 is configured to determine a question-answer result of the query statement according to the candidate subgraph;

[0151] The output module 506 is configured to output the voice of the question and answer result through the playback module.

[0152] Optionally, the second determining module 504 may include:

[0153] A preprocessing submodule is configured to preprocess each of the subgraphs to obtain a subgraph sentence corresponding to the subgraph, and form a sentence pair with the subgraph sentence and the query sentence;

[0154] A first input submodule is configured to input the sentence pair into the semantic matching model to obtain a first matching degree of the sentence pair output by the semantic matching model, wherein the semantic matching model is obtained by training with sample sentence pairs as model input parameters and the first sample matching degrees of the sample sentence pairs as model output parameters;

[0155] The first determination submodule is configured to determine the subgraphs corresponding to the K sentence pairs with the highest first matching degree as candidate subgraphs, where K is greater than 1 and less than or equal to the number of all subgraphs.

[0156] Optionally, the third determining module 505 may include:

[0157] A second determination submodule is configured to determine a target subgraph according to the candidate subgraphs and a preset reordering model;

[0158] The third determination submodule is configured to determine the question and answer result of the query statement according to the target subgraph.

[0159] Optionally, the second determining submodule may include:

[0160] An acquisition submodule, configured to acquire relevant feature data of each of the candidate subgraphs;

[0161] A first input submodule is configured to input, for each of the candidate subgraphs, relevant feature data of the candidate subgraph into the re-ranking model to obtain a second matching degree between the candidate subgraph output by the re-ranking model and the query statement, wherein the re-ranking model is obtained by training with sample subgraphs as model input parameters and the second sample matching degrees of the sample subgraphs as model output parameters;

[0162] The fourth determining submodule is configured to determine the candidate subgraph with the second highest matching degree as a target subgraph.

[0163] Optionally, the relevant feature data includes at least one of the following: a first matching degree of the sentence pair where the candidate subgraph is located, a third matching degree between the candidate subgraph and the query statement, first feature data used to characterize whether specified characters of the query statement exist in the candidate subgraph, and second feature data used to characterize whether the answer type of the question word of the query statement is consistent with the answer type of the candidate subgraph.

[0164] Optionally, the third matching degree is determined in the following manner:

[0165] Respectively determine the first character length of the longest matching character between the entity mention word of the entity in the candidate subgraph and the query sentence, the second character length of the longest matching character between the attribute and the query sentence, and the third character length of the longest matching character between the attribute value and the query sentence, and sum the first character length, the second character length, and the third character length to obtain a fourth character length L;

[0166] Sum the character lengths of entity mentions, attributes, and attribute values ​​of the entities in the candidate subgraph to obtain a fifth character length M;

[0167] The ratio of the fourth character length L to the fifth character length M is determined as the third matching degree between the candidate subgraph and the query statement.

[0168] Optionally, the acquisition module 502 may include:

[0169] a fifth determination submodule, configured to, when determining that the voice information is a query sentence, determine a mentioned word of the query sentence in the mentioned word set corresponding to the knowledge graph as a target mentioned word;

[0170] The sixth determination submodule is configured to query a preset mapping relationship table to obtain all preset object texts of the target mention word for each of the target mention words if the target mention word is an entity mention word; if the target mention word is a string attribute mention word, the target mention word is determined as the object text, wherein the mapping relationship table is used to characterize the correspondence between the entity mention word and the object text.

[0171] Optionally, the set of mentioned words corresponding to the knowledge graph is determined in the following manner:

[0172] Get all entity and string attribute values ​​in the knowledge graph;

[0173] Obtaining entity mentions from the entity, and obtaining string attribute mentions from the string attribute value;

[0174] The entity mentioned words and the string attribute mentioned words are used to construct a mentioned word set corresponding to the knowledge graph.

[0175] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0176] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the voice interaction method for the smart product provided by the present disclosure.

[0177] Figure 6 800 is a block diagram of a voice interaction device for a smart product according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0178] Reference Figure 6 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0179] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the voice interaction method based on the smart product. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0180] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0181] The power component 806 provides power to the various components of the device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0182] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0183] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0184] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0185] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0186] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0187] In an exemplary embodiment, the device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the voice interaction method of smart products.

[0188] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the above instructions can be executed by the processor 820 of the device 800 to complete the voice interaction method of the smart product. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0189] In another exemplary embodiment, a computer program product is also provided, which includes a computer program that can be executed by a programmable device, and the computer program has a code portion for executing the voice interaction method of the above-mentioned smart product when executed by the programmable device.

[0190] Figure 7 1 is a block diagram of a voice interaction device 1900 of a smart product according to an exemplary embodiment. For example, the device 1900 may be provided as a server. Figure 7 , the device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the voice interaction method of the smart product.

[0191] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output (I / O) interface 1958. The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0192] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0193] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.< / answer> < / answer> < / answer>

Claims

1. A voice interaction method for an intelligent product, characterized in that: include: Collect voice information through the collection module; When it is determined that the voice information is a question sentence, obtaining all preset object texts in the question sentence; Determine all subgraphs corresponding to the query statement according to all the preset object texts and the preset path template; Determine a candidate subgraph according to all the subgraphs and a preset semantic matching model; Determine a question-answer result of the query statement according to the candidate subgraph; The voice of the question and answer result is outputted through the playback module.

2. The method according to claim 1, characterized in that The step of determining a candidate subgraph according to all the subgraphs and a preset semantic matching model includes: For each of the subgraphs, preprocess the subgraph to obtain a subgraph sentence corresponding to the subgraph, and form a sentence pair with the subgraph sentence and the query sentence; Inputting the sentence pair into the semantic matching model to obtain a first matching degree of the sentence pair output by the semantic matching model, wherein the semantic matching model is obtained by training with sample sentence pairs as model input parameters and the first sample matching degrees of the sample sentence pairs as model output parameters; The subgraphs corresponding to the K sentence pairs with the highest first matching degrees are determined as candidate subgraphs, where K is greater than 1 and less than or equal to the number of all subgraphs.

3. The method according to claim 1, characterized in that The determining the question-answer result of the query statement according to the candidate subgraph includes: Determine a target subgraph according to the candidate subgraphs and a preset reordering model; The question-answer result of the query statement is determined according to the target subgraph.

4. The method according to claim 3, characterized in that The step of determining a target subgraph according to the candidate subgraphs and a preset reordering model includes: Obtaining relevant feature data of each of the candidate subgraphs; For each of the candidate subgraphs, inputting relevant feature data of the candidate subgraph into the re-ranking model to obtain a second matching degree between the candidate subgraph output by the re-ranking model and the query statement, wherein the re-ranking model is obtained by training with sample subgraphs as model input parameters and the second sample matching degrees of the sample subgraphs as model output parameters; The candidate subgraph with the second highest matching degree is determined as the target subgraph.

5. The method according to claim 4, characterized in that The relevant feature data includes at least one of the following: a first matching degree of the sentence pair where the candidate subgraph is located, a third matching degree between the candidate subgraph and the query statement, a first feature data used to characterize whether a specified character of the query statement exists in the candidate subgraph, and a second feature data used to characterize whether the answer type of the question word of the query statement is consistent with the answer type of the candidate subgraph.

6. The method according to claim 5, characterized in that The third matching degree is determined by: Respectively determine a first character length of the longest matching character between an entity mention of an entity in the candidate subgraph and the query, a second character length of the longest matching character between an attribute and the query, and a third character length of the longest matching character between an attribute value and the query, and sum the first character length, the second character length, and the third character length to obtain a fourth character length L; Sum the character lengths of entity mentions, attributes, and attribute values ​​of the entities in the candidate subgraph to obtain a fifth character length M; The ratio of the fourth character length L to the fifth character length M is determined as a third matching degree between the candidate subgraph and the query statement.

7. The method according to any one of claims 1 to 6, characterized in that When determining that the voice information is a question sentence, obtaining all preset object texts in the question sentence includes: When it is determined that the voice information is a query sentence, a mentioned word of the query sentence in the mentioned word set corresponding to the knowledge graph is determined as a target mentioned word; For each of the target mentioned words, if the target mentioned word is an entity mentioned word, a preset mapping relationship table is queried to obtain all preset object texts of the target mentioned word; if the target mentioned word is a string attribute mentioned word, the target mentioned word is determined as the object text, wherein the mapping relationship table is used to characterize the correspondence between the entity mentioned word and the object text.

8. The method according to claim 7, characterized in that The set of mentioned words corresponding to the knowledge graph is determined in the following way: Get all entity and string attribute values ​​in the knowledge graph; Obtaining entity mentions from the entity, and obtaining string attribute mentions from the string attribute value; The entity mentioned words and the string attribute mentioned words are used to construct a mentioned word set corresponding to the knowledge graph.

9. A voice interaction device for an intelligent product, characterized in that: include: A collection module, configured to collect voice information through a collection module; An acquisition module, configured to acquire all preset object texts in the inquiry sentence when determining that the voice information is an inquiry sentence; A first determination module is configured to determine all subgraphs corresponding to the query statement according to all the preset object texts and the preset path template; A second determination module is configured to determine a candidate subgraph according to all the subgraphs and a preset semantic matching model; A third determination module is configured to determine a question-answer result of the query statement according to the candidate subgraph; The output module is configured to output the voice of the question and answer result through the playback module.

10. A voice interaction device for an intelligent product, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: Collect voice information through the collection module; When it is determined that the voice information is a question sentence, obtaining all preset object texts in the question sentence; Determine all subgraphs corresponding to the query statement according to all the preset object texts and the preset path template; Determine a candidate subgraph according to all the subgraphs and a preset semantic matching model; Determine a question-answer result of the query statement according to the candidate subgraph; The voice of the question and answer result is outputted through the playback module.

11. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images