A method for building a personalized intelligent assistant based on general knowledge graph

Abstract

The present invention relates to a method for building a personalized intelligent assistant based on a general knowledge graph. The method introduces a general knowledge graph into the scenario modeling and development process, and realizes the automatic matching and execution of user natural language instructions to APIs through API matching and recommendation based on similarity calculation and API parameter extraction based on feature matching.

Description

Technical Field

[0001] The present invention relates to a method for constructing a personalized intelligent assistant based on a general knowledge graph. Background Art

[0002] With the rise and development of artificial intelligence technology, the emergence of intelligent assistants has brought people a new human-computer interaction experience. Intelligent services have been further published as service interfaces (Application Programming Interface, API). Users can complete the call of services only through natural language instructions, which optimizes complex operations and saves time. However, in real application scenarios, there are always some problems in the call of intelligent services. On the one hand, intelligent assistants need to follow the natural language mapping and parameter positioning rules pre-defined in the system. Once the user's natural language instructions are significantly different from the rules, the corresponding API cannot be matched and executed, and the redefinition of the rules can only be achieved by modifying the underlying logic. On the other hand, most of the current intelligent assistants annotate and then identify massive natural language instructions. When the number of samples added by users is too small, it is impossible to train intelligent services according to personal usage habits. Therefore, how to define a natural language mapping method and provide end users with a personalized intelligent assistant that can quickly create and modify service settings based on changing needs is of great significance and value. Summary of the invention

[0003] The purpose of the present invention is to provide a method for building a personalized intelligent assistant based on a general knowledge graph, which can realize the automatic matching and execution of user natural language instructions to API.

[0004] To achieve the above object, the technical solution of the present invention is: a method for constructing a personalized intelligent assistant based on a general knowledge graph, comprising:

[0005] Feature model construction based on general knowledge graph: Based on the structural and semantic features of the instructions, natural language instruction features are constructed with the help of structured data of general knowledge graph, and then API features are constructed to represent scene features. On this basis, a feature extraction method based on general knowledge graph is proposed;

[0006] API matching and recommendation based on similarity calculation: Based on the constructed feature model, the triple relationship similarity and label similarity of the user instructions are calculated, and the API that best matches the user instructions is obtained based on the weighted results of the two.

[0007] API parameter extraction method based on feature matching: Based on the constructed feature model, the user instructions are labeled and vectorized for the matched API, and the instruction slot values ​​are extracted based on the labels and prepositions to obtain the parameters required to execute the API and complete the API execution.

[0008] In one embodiment of the present invention, the specific implementation process of constructing the feature model based on the general knowledge graph is as follows:

[0009] (1) Natural language instruction features

[0010] The natural language instruction features include the following four features: natural language instruction triple set TripleSet, instruction keyword set KeySet, instruction preposition set PrepSet and natural language instruction label set LabelSet; define a natural language instruction as comd; among them,

[0011] (1.1) TripleSet of natural language instruction triples

[0012] For each natural language instruction, extract the instruction triplet t i After that, a triple set comd.TripleSet = {t1, t2, ..., t n};

[0013] (1.2) Command keyword KeySet

[0014] The keyword KeySet of the command definition is composed of the nouns in the command and the named entity recognition results: comd.KeySet = (N, Spe) = {k1, k2, ..., k n}, where N is the set of nouns in the example instruction instance, Spe is the words whose named entity recognition annotation is not empty, and k i For keywords;

[0015] (1.3) Instruction preposition set PrepSet

[0016] If the keyword k i The preceding word is identified as a preposition and recorded as the keyword preposition k i .prep, so the command preposition set comd.PrepSet = {<k1.prep,k1>, <k2.prep,k2>..., <k n .prep,k n >}, is a set of binary relationship groups for each keyword and its preposition;

[0017] (1.4) Natural language instruction label set LabelSet

[0018] The label set of a word w can be expressed as w.LabelSet = {l1,l2,...,l n}, then the natural language instruction label set can be expressed as the union of each keyword label set, comd.LabelSet = {k1.LabelSet, k2.LabelSet, ..., k n .LabelSet}={l1,l2,...,l m};

[0019] (2) API Features

[0020] Define an API as API = {APIName, Instance, ParaName}, where APIName is the application interface name, Instance = {instance1, instance2, ..., instance n} are n natural language example instructions for calling the API, ParaName = {slot1, slot2, ..., slot m} is the m parameters required to execute the API. This parameter restriction is also called a word slot. The parameter value required to execute the API is the slot value. n .value; Similar to the natural language instruction features, the API features are defined as the structural features and semantic features of the example instructions for calling the API; therefore, the API features include three elements: API triple set TripleSet, API label set KeySet, and API slot label set LabelSet;

[0021] (2.1) API Configuration

[0022] Users can add x API configuration files as needed to meet their service scenarios; for each application program interface api i , users can personalize n service natural language examples Instance for it, and configure m word slots ParaName of the API; for each example instance j , configure the slot value corresponding to the API parameter in its natural language instruction n .value;

[0023] (2.2) API triple set

[0024] API triple collection api i .TripleSet contains a set of instruction triples for each example in the API. j .TripleSet;

[0025] (2.3) API tag set

[0026] Calculate all instances j The frequency of each label in .LabelSet i .f, if l i .f>F, then record l i is the core tag set of the API, and the calculation formula of F is as follows:

[0027] F=w%×n

[0028] API tag collection api i .LabelSet contains n examples of the service j .All core tags of KeySet;

[0029] (2.4) API Slot Tag Set

[0030] Use the general knowledge graph to interpret each set word slot of the API and obtain the API word slot label set; the API has m word slots that need to be filled. For multiple examples of an API, obtain their same word slot j Different slot values ​​under slot j .value tag set, after high-frequency processing of different slot value tag sets of the same word slot, the tag set api of the word slot is obtained i .ParaName.LabelSet; thereby obtaining the label set api for different word slots in the API i .slot j .LabelSet.

[0031] In one embodiment of the present invention, the implementation method of the API matching and recommendation based on similarity calculation is: after obtaining the feature models of user instructions and APIs in multiple scenarios, the triple relationship similarity tri_sim and label similarity lab_sim are used to perform similarity calculation to match the API; the triple relationship similarity tri_sim represents the action intention of two events, and the action intentions of the two events can be judged by matching the relationship similarity; the label similarity lab_sim can judge whether the semantics of the two events are similar by calculating the label similarity; therefore, using the triple relationship similarity tri_sim and the label similarity lab_sim, the similarity API_Sim between the user instruction and the API in each scenario is obtained through weighted calculation, and the similarity result Top1 is taken as the matched API, and the scene matching result of the user instruction is returned.

[0032] In one embodiment of the present invention, the specific implementation process of the API matching and recommendation based on similarity calculation is as follows:

[0033] (1) API similarity calculation based on triples

[0034] The GloVe model trained with large-scale training corpus is used to represent the vocabulary in natural language text in a distributed manner. The vector of a sentence can be represented by the average of the vectors of all predicate words in the sentence. The sentence is converted into sentence predicate vectors with the same length for representation. The GloVe vector of each word w can be represented as an m-dimensional vector vec(w)=(x i1 ,x i2 ,...,x im );

[0035] Define the sentence predicate vector V as the average of the word vectors of all relations with the same dimension, then the sentence can also be represented by an m-dimensional vector V, V = (V1, V2, ..., V m ), V j The calculation formula is as follows:

[0036]

[0037] If the natural language instruction input by the user is ucmd, then based on the word embedding technology, the user instruction triple set ucmd.TripleSet is vectorized; first, all triple instances of each instruction are traversed j .TripleSet, for each triple, take out each TripleSet.relation and add it to the triple relation set, ucmd.RelationSet = {relation1,relation2,...,relation m}; query the vector of each word in the Glove model; for a phrase, take the average to get the vector of the phrase; take the average of the vectors of all phrases to get the triple relationship vector TriVec of the entire instruction; the triple relationship vector of the user instruction is ucmd.TriVec=(utv1,utv2,...utv m );

[0038] Define API.relation as an API relation set, which contains the triple relations of all instructions in n examples of the API; instance j .RelationSet={r1,r2,…,r n}; where r i For the example instruction instance j .TripleSet each triple Ti T i .relation; In the same way, based on the Glove model, we get the triple relation vector instance of each example j .TriVec, sum and average each example vector of the API to obtain the triple relationship of each API i .TriVec=(atv1,atv2,…,atv m );

[0039] After finding the angle between the two sentence vectors, the cosine value can be used to determine the similarity of the individuals; sentence a is represented by a vector (x1, x2, …, x m ), sentence b is represented by a vector (y1,y2,…,y m ); then the similarity of two multidimensional vectors can be calculated as follows:

[0040]

[0041] Use similarity calculation to compare the user instruction triple relationship vector ucmd.TriVec with the API triple relationship vector api i .TriVec uses the similarity calculation algorithm to obtain tri_sim, which is the similarity of the triple relationship between the user instruction and each API; the calculation formula is as follows:

[0042]

[0043] When the value of tri_sim is closer to 1, it means that the vector of the user input command is more similar to the vector of the API, which means that the action intention of the application scenario provided by the API is closest to the user action intention;

[0044] (2) API similarity calculation based on tags

[0045] (2.1) Label vectorization for APIs

[0046] The label vectorization process refers to the process of vectorizing two label sets. Through vectorization, the similarity between the two label sets can be calculated, so as to determine the similarity between user instructions and APIs. For the label set, the bag of words model (BOW) is used to complete the vectorization. The bag of words model assumes that the word order, grammar and syntax of a text are ignored, and it is only regarded as a set of words or a combination of words. The appearance of each word in the text is independent and does not depend on whether other words appear.

[0047] The user instruction needs to calculate the similarity with multiple APIs respectively to determine which API scenario the instruction is closest to. Therefore, the user instruction label set needs to be vectorized for each API to determine the overlap between the two label sets.

[0048] Assume that each API label set is a vector consisting of only 1s;

[0049] API i .LabelVec=(alv1,alv2,…,alv m )=(1,1,…,1)

[0050] Input the feature models of all APIs and the feature models of user natural language instructions; according to the bag-of-words model, for each API, determine whether the words that appear in the user's natural language instruction tag set appear in the API tag set. If so, set it to 1, otherwise set it to 0; finally, for n APIs i Get n vectors containing 0 and 1 ucmd.LabelVec i ; Output the vector results of the user instructions for each API:

[0051] ucmd.LabelVec i =(ulv1,ulv2,…,ulv m ) ulv m ∈N∧ulv m ∈[0,1]

[0052] (2.2) Tag-based API matching method

[0053] By passing the user specified label vector ucmd.LabelVec i With all API tags vector api i .LabelVec performs similarity calculation to obtain the similarity lab_sim between the user instruction and the label set of each API. The calculation formula is as follows:

[0054]

[0055] (2.3) API recommendation similarity calculation

[0056] The similarity between user instructions and APIs in each scenario, API_sim, takes into account two features: one is the structural triple feature, which can represent the action intention of an instruction through the relationship in the triple and distinguish different purposes; the other is the semantic label set feature, which adds label interpretations to the API through the general knowledge graph to represent the semantics of the API; therefore, API_sim is calculated by the triple relationship similarity tri_sim and the label set similarity lab_sim, and the formula is as follows, where α is the empirical value, α∈R + ∧α∈(0,1);

[0057] API_sim=α×tri_sim+(1-α)×lab_sim

[0058] The above formula can be used to calculate the recommended similarity API_sim between the user command and each API. The closer the value of API_sim is to 1, the closer the semantics expressed by the two are. Therefore, according to the descending order, the API with the top 1 API similarity is obtained, and this API is returned as the final matched scenario.

[0059] In one embodiment of the present invention, the implementation method of the feature matching-based API parameter extraction method is as follows: the API word slot label similarity KeyLabel_sim and the preposition similarity KeyPrep_sim are used to extract and match keywords, and the word slot similarity Key_Sim is obtained through weighted calculation; the parameters required for API execution are obtained from user instructions according to the similarity sorting of the word slot similarity Key_Sim.

[0060] In one embodiment of the present invention, the specific implementation process of the API parameter extraction method based on feature matching is as follows:

[0061] (1) For each keyword label vector of the user instruction and each word slot label vector of the API, the label similarity KeyLabel_sim is calculated using cosine similarity. The calculation formula is as follows:

[0062]

[0063] (2) Calculate the similarity between prepositions KeyPrep_sim

[0064] Use the Glove model to vectorize each preposition word and vectorize each keyword k in the user instruction. i The corresponding preposition k1.prep is taken from the preposition set and vectorized based on the Glove model as k i .PrepVec=(upv1,upv2,...,upv m );

[0065] The prefix of the API slot value is the prefix set of the API example instruction; find the slot value slot in the API example j .value, extract the word slot preposition value.prep, the vectorized representation result of the API preposition is slot j .value.PrepVec=(apv1,apv2,...,apv m );

[0066] Use cosine similarity to calculate the similarity KeyPrep_sim between the keyword preposition in the user command and the API word slot preposition vector; the calculation formula is as follows:

[0067]

[0068] (3) Calculate the similarity between the user command keywords and the word slots

[0069] Use weight calculation to get the word slot similarity Key_sim; calculate which keyword in the user instruction is closest to the meaning of the word slot, so as to find the closest keyword as the slot value of the word slot and fill it into the API; the formula is as follows, where β is the experience value, β∈R + ∧β∈(0,1);

[0070] Key_sim=β×KeyLabel_sim+(1-β)×KeyPrep_sim

[0071] Sorting is performed in descending order according to the word slot similarity, and the slot value corresponding to a word slot in the user instruction is obtained from the high to low similarity, that is, the parameters required to execute the API are extracted from the user instruction, and a word slot matching list is obtained, that is, the parameters required to execute the API are obtained to complete the API execution.

[0072] Compared with the prior art, the present invention has the following beneficial effects: the method of the present invention introduces a general knowledge graph into the scenario modeling and development process, and realizes the automatic matching and execution of user natural language instructions to APIs through API matching and recommendation based on similarity calculation and API parameter extraction method based on feature matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 This is an overview of the method for building a personalized intelligent assistant based on a general knowledge graph in the present invention.

[0074] Figure 2 Matching flow chart for API.

[0075] Figure 3 It is a user instruction tag set and an API tag set.

[0076] Figure 4 Extract method flow for API parameters.

[0077] Figure 5 Keywords and API word slots for user instructions. DETAILED DESCRIPTION

[0078] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0079] The present invention provides a method for constructing a personalized intelligent assistant based on a general knowledge graph, comprising:

[0080] Feature model construction based on general knowledge graph: Based on the structural and semantic features of the instructions, natural language instruction features are constructed with the help of structured data of general knowledge graph, and then API features are constructed to represent scene features. On this basis, a feature extraction method based on general knowledge graph is proposed;

[0081] API matching and recommendation based on similarity calculation: Based on the constructed feature model, the triple relationship similarity and label similarity of the user instructions are calculated, and the API that best matches the user instructions is obtained based on the weighted results of the two.

[0082] API parameter extraction method based on feature matching: Based on the constructed feature model, the user instructions are labeled and vectorized for the matched API, and the instruction slot values ​​are extracted based on the labels and prepositions to obtain the parameters required to execute the API and complete the API execution.

[0083] The following is the specific implementation process of the present invention.

[0084] Figure 1 This is an overview of the method for building a personalized intelligent assistant based on a general knowledge graph. The method of the present invention introduces a general knowledge graph into the scene modeling and development process, and realizes the automatic matching and execution of user natural language instructions to APIs through API matching and recommendation based on similarity calculation and API parameter extraction based on feature matching. For personalized intelligent assistants, the main work of this system has three parts:

[0085] (1) Feature model construction based on general knowledge graph: Based on the structural features and semantic features of the instructions, the natural language instruction features are constructed with the help of the structured data of the general knowledge graph, and then the API features are constructed to represent the scene features. On this basis, a feature extraction method based on the general knowledge graph is proposed. The feature model can understand the semantics of natural language instructions and the semantics of APIs.

[0086] (2) API matching and recommendation based on similarity calculation: Based on the constructed semantic model, the triple relationship similarity and label similarity of user instructions are calculated, and the API that best matches the user instruction is obtained based on the weighted results of the two.

[0087] (3) API parameter extraction method based on feature matching: Based on the constructed feature model, the user instructions are labeled and vectorized for the API matched in (2), and the instruction slot values ​​are extracted based on the labels and prepositions to obtain the parameters required to execute the API and complete the API execution.

[0088] 1. API construction method based on general knowledge graph

[0089] 1.1. Natural language instruction features

[0090] Natural language instruction features often include structural features and semantic features. Triple extraction refers to extracting structured relationship triples from plain text, which represent the relationship between two entities and entities. i =<subject,relation,object>, where subject is the subject of the sentence, relation is the predicate, indicating the relationship between the subject and the object, and object is the object. The triple relationship reflects the characteristics of the instruction predicate, and can often express the action intention of the user instruction. Therefore, triples are used to reflect the structural characteristics of natural language instructions. Keywords are often the most critical part of a sentence or chapter, which can reflect the core idea of ​​the sentence or chapter and contain a lot of information. Prepositions are words or affixes used to express the grammatical function of a word, and form prepositional structures with nouns to indicate place, time, method, etc. This system obtains service labels through the general knowledge graph Wikidata. The properties in Wikidata represent various affiliations (A-Kind-of, Is-A, Part-of relationships). The value (Property Value) containing multiple attributes can find a subject word with a broader extension in the concept of the word as the label of the word, thereby representing the basic characteristics of a keyword or phrase. Therefore, this system uses triples, keywords, prepositions, and tag sets to reflect the semantic features of natural language instructions.

[0091] Therefore, the natural language instruction features include the following four features: natural language instruction triples TripleSet, instruction keyword sets KeySet, prepositions PrepSet and natural language instruction labels LabelSet.

[0092] Define a natural language instruction as comd.

[0093] (1) TripleSet

[0094] When extracting natural language features, the StanfordCoreNLP tool (https: / / stanfordnlp.github.io / CoreNLP / ) is used for natural language processing. The tool can perform triple extraction, named entity recognition, part-of-speech tagging, and other functions. For each natural language instruction, the instruction triple t i After that, a triple set comd.TripleSet = {t1, t2, ..., t n}. Since the expressions of instructions are complex and diverse, a sentence usually contains multiple triples.

[0095] (2) Command keyword KeySet

[0096] The keywords contained in each natural language instruction are usually composed of multiple words. It contains words that can represent the special characteristics of the instruction, as well as the results of named entity recognition by the tool. Usually, the nouns in the instruction can represent the special characteristics of the instruction. Named entity recognition can make a preliminary judgment on the entity type and can recognize the special semantics of types such as LOCATION, DATE, and TIME. The keyword KeySet that defines the instruction is composed of the nouns in the instruction and the results of named entity recognition. comd.KeySet=(N,Spe)={k1,k2,...,k n}. N is the set of nouns in the example instruction instance. Spe is the word whose named entity recognition annotation is not empty.

[0097] (3) Instruction preposition set PrepSet

[0098] Based on the part-of-speech tagging of the StanfordCoreNLP tool, if the keyword k i The preceding word is identified as a preposition and recorded as the keyword preposition k i .prep. Through the tool extraction, we can get the instruction preposition set comd.PrepSet = {<k1.prep,k1>, <k2.prep,k2>..., <k n .prep,k n >}, is a set of binary relationship groups for each keyword and its preposition.

[0099] (4) Instruction label set LabelSet

[0100] The label set of a word w can be expressed as w.LabelSet = {l1,l2,...,l nThen the label set of the natural language instruction can be expressed as the union of each keyword label set, comd.LabelSet = {k1.LabelSet, k2.LabelSet, ..., k n .LabelSet}={l1,l2,...,l m}.

[0101] Based on the Wikidata open source knowledge graph, a tag set of a word can be obtained. In Wikidata, each word has multiple properties and corresponding attribute values. When the attribute is a subordination relationship (A-Kind-of, Is-A, Part-of relationship), the word with a broader conceptual extension can be found through the attribute value, thereby expressing the semantics of a word. In Wikidata, subordination is expressed as subclass of (A-Kind-of, Part-of relationship) and instance of (Is-A relationship).

[0102] Through the Wikidata data interface, we can obtain multiple attributes related to word w and the relation triple S=<w,s i ,l i >. Among them, s i Indicates the attribute relationship between words. i =instance of, it means l i is one of the instances of the word. i = subclass of, it means that it includes Is-A, A-Kind-of or Part-of, and then l i As the attribute label of the word. Through iteration, multiple attribute values ​​under the word affiliation are found and recorded as a label set.

[0103] The process of obtaining the attribute tag set of word w is as follows:

[0104] Step 1: For word w, use StanfordCoreNLP tool to perform named entity recognition. If it is not empty, the recognition result is recorded as l i , will l i Added w.LabelSet.

[0105] Step 2: Access the Wikidata open source knowledge graph. i =subclass of, through the triple <w,subclass of,l i > Find w is l i A subclass of l i is one of the attribute tags of the word, and li Added w.LabelSet.

[0106] Step 3: If the word does not exist s i =subclass of, then it means that the word is a specific instance.

[0107] Take i =instance of, through the triple <w,instance of,l i > Found the word is l i For example, let w = l i , repeat step 2.

[0108] Step 4: Get w.LabelSet = {l1,l2,...,l n}.

[0109] Repeat the acquisition process for the key semantic set of the user command to obtain the label set comd.LabelSet of the user command.

[0110] 1.2. Programming language features

[0111] APIs are predefined interfaces in programming languages, and their purpose is to provide services required in certain scenarios. APIs can meet the functions required to complete user instructions. Define an API as API = {APIName, Instance, ParaName}. Among them, APIName is the name of the application interface, Instance = {instance1, instance2, ..., instance n} are n natural language example instructions for calling the API, ParaName = {slot1, slot2, ..., slot m} is the m parameters required to execute the API. This parameter restriction is also called a slot. The parameter value required to execute the API is the slot value. n .value. Similar to the natural language instruction features, the API features are defined as the structural features and semantic features of the example instructions for calling the API. Therefore, the API features include three elements: API triples TripleSet, API label set KeySet, and API word slot label set LabelSet.

[0112] (1) API configuration

[0113] Users can add x API configuration files as needed to meet their service scenarios. i, users can customize n service natural language examples Instance for it, and configure m word slots ParaName of the API. j , configure the slot value corresponding to the API parameter in its natural language instruction n .value.

[0114] (2) API triple set

[0115] API triple collection api i .TripleSet contains the instruction triple set instance for each example in the API j .TripleSet.

[0116] (3) API tag set

[0117] In order to remove the influence of some irrelevant labels on the central semantics, it is necessary to perform high-frequency processing on the label sets of different instructions in multiple examples. The common labels that appear in more than w% of the label sets are recorded as core labels. The process of high-frequency processing is: calculate all instance j The frequency of each label in .LabelSet i .f, if l i .f>F, then record l i is the core tag set of the API. The calculation formula of F is as follows.

[0118] F=w%×n

[0119] API tag collection api i .LabelSet contains n examples of the service j All core tags of .KeySet.

[0120] (4) API slot tag set

[0121] Use the general knowledge graph to interpret each set word slot of the API and obtain the API word slot label set. The API has m word slots that need to be filled. For multiple examples of an API, obtain their same word slot j Different slot values ​​under slot j .value tag set, after high-frequency processing of different slot value tag sets of the same word slot, the tag set api of the word slot is obtained i .ParaName.LabelSet.

[0122] This obtains the label set api for different word slots in the API i .slot j.LabelSet.

[0123] 2. API matching and recommendation based on similarity calculation

[0124] The present invention proposes an API matching and recommendation method based on similarity calculation. After obtaining the feature models of user instructions and APIs in multiple scenarios, the triple relationship similarity tri_sim and label similarity lab_sim are used to perform similarity calculation to match the API. The triple relationship similarity tri_sim represents the action intention of two events. By matching the relationship similarity, it can be determined whether the action intentions of the two events are similar. The label similarity lab_sim can determine whether the semantics of two events are similar by calculating the label similarity. Therefore, using the above two factors, the similarity API_Sim between the user instruction and the API in each scenario is obtained through weighted calculation, and the similarity result Top1 is taken as the matched API, and the scene matching result of the user instruction is returned. The process is as follows Figure 2 .

[0125] 2.1. API Similarity Calculation Based on Triplet

[0126] The natural language used in daily life cannot be directly understood by computers. When users need to process these natural languages, they need to use specific means to analyze or preprocess them. Using continuous vectors, words can be represented by numerical values, thereby representing the relationship between words and instructions.

[0127] The word embedding technology can be used to realize the digitization of text. The method of the present invention uses the Glove model trained by large-scale training corpus to distribute the vocabulary in the text. The vector of a sentence can be represented by the average value of the sum of all predicate word vectors in the sentence. The sentence is converted into a sentence predicate vector with the same length by the above method for representation. The Glove vector of each word w can be represented as an m-dimensional vector vec(w)=(x i1 ,x i2 ,…,x im ).

[0128] Define the sentence predicate vector V as the average of the word vectors of all relations with the same dimension, then the sentence can also be represented by an m-dimensional vector V, V = (V1, V2, ..., V m ). j The calculation formula is as follows.

[0129]

[0130] If the natural language instruction input by the user is ucmd, then based on the word embedding technology, the user instruction triple set ucmd.TripleSet is vectorized. First, all triple instances of each instruction are traversed. j .TripleSet, for each triple, take out each TripleSet.relation and add it to the triple relation set, ucmd.RelationSet = {relation1,relation2,...,relation m}. Query the vector of each word in the Glove model. For a phrase, take the average to get the vector of the phrase. Take the average of the vectors of all phrases to get the triple relationship vector TriVec of the entire instruction. The triple relationship vector of the user instruction is ucmd.TriVec=(utv1,utv2,…utv m ).

[0131] Define API.relation as an API relation set, which contains the triples of all instructions in the n instances of the API. j .RelationSet={r1,r2,…,r n}. Among them, r i For the example instruction instance j .TripleSet each triple T i T i In the same way, based on the Glove model, we get the triple relation vector instance of each example. j .TriVec, sum and average each example vector of the API to obtain the triple relationship of each API i .TriVec=(atv1,atv2,…,atv m ).

[0132] Cosine similarity uses the cosine value of the angle between two vectors in the vector space as a measure of the size of the difference between two individuals. The closer the cosine value is to 1, the closer the angle is to 0°, that is, the more similar the two vectors are. This is called "cosine similarity". From the above conclusion, it can be seen that after obtaining the angle between the two sentence vectors, the cosine value can be used to determine the similarity of individuals. Sentence a is represented by a vector (x1, x2, ..., x m ), sentence b is represented by a vector (y1,y2,…,y m ). Then the similarity of two multidimensional vectors can be calculated by the following formula.

[0133]

[0134] Use similarity calculation to compare the user instruction triple relationship vector ucmd.TriVec with the API triple relationship vector api i .TriVec uses the algorithm for similarity calculation, and tri_sim is the similarity of the triple relationship between the user instruction and each API. The calculation formula is as follows.

[0135]

[0136] When the value of tri_sim is closer to 1, it means that the vector of the user input instruction is more similar to the vector of the API, which means that the action intention of the application scenario provided by the API is closest to the user action intention.

[0137] 2.2. API Similarity Calculation Based on Tags

[0138] 2.2.1. Label vectorization for APIs

[0139] The label vectorization process refers to the process of vectorizing two label sets. Through vectorization, the similarity between two label sets can be calculated, so as to determine the similarity between user instructions and APIs. For label sets, the bag of words model (BOW) is used to complete vectorization. The bag of words model is a simplified expression model in natural language processing and information retrieval, which is used for text vectorization. The bag of words model assumes that the word order, grammar and syntax of a text are ignored, and it is only regarded as a set of words or a combination of words. The appearance of each word in the text is independent and does not depend on whether other words appear.

[0140] The similarity between user instructions and multiple APIs should be calculated separately to determine which API scenario the instruction is closest to. Therefore, the user instruction label set should be vectorized for each API to determine the degree of overlap between the two label sets.

[0141] Let each API label set be a vector consisting of only 1s.

[0142] API i .LabelVec=(alv1,alv2,…,alv m )=(1,1,…,1)

[0143] Input the feature models of all APIs and the feature models of user natural language instructions. According to the bag-of-words model, for each API, determine whether the words that appear in the user's natural language instruction tag set appear in the API tag set. If so, set it to 1, otherwise set it to 0. Finally, for n APIs, i Get n vectors containing 0 and 1 ucmd.LabelVeci . Output the vector results of the user instructions for each API.

[0144] ucmd.LabelVec i =(ulv1,ulv2,…,ulv m ) ulv m ∈N∧ulv m ∈[0,1]

[0145] like Figure 3 As shown in the figure, the label set of the user's natural language instructions needs to be vectorized for each API's label set. The label set of the user instruction "Set an alarm at 8 o'clock." is {'device', 'signal', 'number'}. The vector of the API for purchasing beverages is (1,1,1,1,...,1), and the vector generated by the user instruction for the API for purchasing beverages is (1,0,0,0,...,0); the vector of the API for querying air tickets is (1,1,1,1,...,1), and the vector generated by the user instruction for querying air tickets is (0,0,0,0,...,0); the vector of the API for setting the alarm is (1,1,1,1,...,1), and the vector generated by the user instruction for setting the alarm is (1,0,1,0,...,1).

[0146] 2.2.2. Tag-based API matching method

[0147] This section uses the cosine similarity method for matching, by using the label vector ucmd.LabelVec i With all API tags vector api i .LabelVec performs similarity calculation to obtain the similarity lab_sim between the user instruction and each API label set. The calculation formula is as follows:

[0148]

[0149] like Figure 3 As shown in Figure 2, from formula (5), we can get that the label set similarities of the user command "Set an alarm at 8 o'clock for me." for the three APIs of buying drinks, querying air tickets, and setting alarms are ('setalarm', 0.5196152422706632), ('buydrink', 0.31819805153394637), ('querymovie', 0.0) respectively. Therefore, among these three example APIs, setting an alarm is closest to the user's command intention.

[0150] 2.2.3. API Recommendation Similarity Calculation

[0151] The similarity between user instructions and APIs in each scenario, API_sim, takes into account two features. One is the structural triple feature, which can represent the action intention of an instruction through the relationship in the triple and distinguish different purposes. The other is the semantic label set feature, which adds label interpretations to the API through the general knowledge graph to represent the semantics of the API. Therefore, API_sim is calculated by the triple relationship similarity tri_sim and the label set similarity lab_sim, and the formula is as follows. Among them, α is the empirical value, α∈R + ∧α∈(0,1)"

[0152] API_sim=α×tri_sim+(1-α)×lab_sim

[0153] Formula (6) can be used to calculate the recommended similarity API_sim between the user command and each API. The closer the value of API_sim is to 1, the closer the semantics expressed by the two are. Therefore, according to the descending order, the API with the top 1 API similarity is obtained, and this API is returned as the final matched scenario.

[0154] 3. API parameter extraction method based on feature matching

[0155] The API matching and recommendation method based on similarity calculation proposed in this invention can return the top 1 similar API matching results, which are the APIs in the required scenario matched by the user instructions. However, the API execution requires filling in parameters, that is, it needs to be in the user instructions, so it is necessary to find the corresponding values ​​of the parameters required to execute the corresponding API. Based on the scenario model construction method in Chapter 4, this chapter proposes an API parameter extraction method based on feature matching. The process is as follows: Figure 4 .

[0156] The API word slot label similarity KeyLabel_sim and preposition similarity KeyPrep_sim are used to extract and match keywords, and the word slot similarity Key_Sim is obtained through weighted calculation. According to the similarity sorting, the parameters required for API execution are obtained from the user instructions.

[0157] 3.1. Parametric feature modeling

[0158] The process of parameter matching is as follows: the keywords extracted from the user instructions are used as candidate parameters in the matching process, and the similarity is calculated with multiple word slots in the API respectively, and the semantics of the candidate parameter and the API word slot are determined to be closest, and the candidate parameter is filled into the word slot as the parameter value. Therefore, each candidate parameter in the user instruction needs to be vectorized for each word slot of the API to determine the degree of overlap of the semantics of the two, so as to find the parameter value in the user instruction and fill it into the API word slot for execution.

[0159] like Figure 5 As shown, there are two word slots that need to be filled in the beverage purchase API: quantity (number) and beverage type (drinkKind). The candidate parameters "two" and "Coca Cola" in the user instruction "I want to buy two cups of Coca Cola." need to be mapped to the corresponding word slots.

[0160] The bag-of-words model is used to vectorize the parameters. The candidate parameters need to be matched with each word slot to determine which word slot is closest to it. Therefore, the word slot label set of the API is first represented as a vector consisting of only 1, and then it is determined whether the word appears in the label set of the candidate parameters. The vectorization result of each word slot label of the API is:

[0161] API i .slot j .LabelVec=(slv1,slv2,…,slv m )=(1,1,...,1)

[0162] The keyword tag set of the user instruction needs to be processed with the multiple word slot tag sets of the API using the bag of words model. For each word slot of the API, determine whether the words that appear in the user instruction keyword tag set appear in the word slot tag set. If so, set it to 1, otherwise set it to 0. The instruction keyword is vectorized for multiple word slots of a certain API to obtain an instruction keyword vector containing 0 and 1 ucmd.LabelVec[api i .slot j ] Returns the vector results of the semantics in the user command for each word slot of the API:

[0163] ucmd.LabelVec[api i .slot j ]=(uslv1,uslv2,...,uslv m ) uslv m ∈N∧uslv m ∈[0,1]

[0164] 3.2. Feature-based API parameter extraction method

[0165] In order to complete the API execution, it is necessary to convert the initial user intention into the parameter information that needs to be completed to clarify the user's instructions, that is, to extract the slot value that can be filled in the corresponding API word slot in the user's natural language instruction, and achieve information correspondence with the word slot.

[0166] Slot value extraction considers two aspects of instruction information. One is the structural preposition feature, which uses prepositions to mark the differences between different word slots. The other is the semantic label feature, which adds labels to different word slots through the general knowledge graph to represent their semantics. Therefore, the word slot similarity Key_sim is composed of the preposition similarity KeyPrep_sim and the label set similarity KeyLabel_sim, and the similarity result of slot value matching is obtained by weighted calculation of the two.

[0167] First, for each keyword label vector of the user instruction and each word slot label vector of the API, the label similarity KeyLabel_sim is calculated using cosine similarity. The calculation formula is as follows:

[0168]

[0169] Secondly, calculate the similarity between prepositions KeyPrep_sim.

[0170] Use the Glove model to vectorize each preposition word. i The corresponding preposition k1.prep is taken from the preposition set and vectorized based on the Glove model as k i .PrepVec=(upv1,upv2,...,upv m ).

[0171] The prefix of the API slot value is the prefix set of the API example instruction. Find the slot value slot in the API example j .value, extract the word slot preposition value.prep, the vectorized representation result of the API preposition is slot j .value.PrepVec=(apv1,apv2,...,apv m ).

[0172] Use cosine similarity to calculate the similarity between the keyword preposition in the user instruction and the API word slot preposition vector KeyPrep_sim. The calculation formula is as follows:

[0173]

[0174] Third, calculate the similarity between the user instruction keywords and the word slots.

[0175] Use weight calculation to get the word slot similarity Key_sim. Calculate which keyword in the user instruction is closest to the meaning of the word slot, so as to find the closest keyword as the slot value of the word slot and fill it into the API. The formula is as follows. Where β is the experience value, β∈R + ∧β∈(0,1).

[0176] Key_sim=β×KeyLabel_sim+(1-β)×KeyPrep_sim

[0177] Sorting in descending order according to the word slot similarity, the slot value corresponding to a word slot in the user instruction is obtained from the high to low similarity, that is, the parameters required to execute the API are extracted from the user instruction, and the word slot matching list is obtained. That is, the parameters required to execute the API are obtained, and the API execution is completed.

[0178] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. A method for constructing a personalized intelligent assistant based on a general knowledge graph, characterized in that: include: Feature model construction based on general knowledge graph: Based on the structural and semantic features of the instructions, natural language instruction features are constructed with the help of structured data of general knowledge graph, and then API features are constructed to represent scenario features; On this basis, a feature extraction method based on general knowledge graph is proposed; API matching and recommendation based on similarity calculation: Based on the constructed feature model, the triple relationship similarity and label similarity of the user instructions are calculated, and the API that best matches the user instructions is obtained based on the weighted results of the two. API parameter extraction method based on feature matching: Based on the constructed feature model, user instructions are labeled and vectorized for the matched API, instruction slot values ​​are extracted based on the labels and prepositions, and the parameters required to execute the API are obtained to complete the API execution; The specific implementation process of constructing the feature model based on the general knowledge graph is as follows: (1) Natural language instruction features The natural language instruction features include the following four features: natural language instruction triple set TripleSet, instruction keyword set KeySet, instruction preposition set PrepSet and natural language instruction label set LabelSet; define a natural language instruction as comd; among them, (1.1) TripleSet of natural language instruction triples For each natural language instruction, extract the instruction triplet t i After that, a triple set comd.TripleSet = {t1, t2, ..., t n }; (1.2) Command keyword KeySet The keyword KeySet of the command definition is composed of the nouns in the command and the named entity recognition results: comd.KeySet = (N, Spe) = {k1, k2, ..., k n }, where N is the set of nouns in the example instruction instance, Spe is the words whose named entity recognition annotation is not empty, and k i For keywords; (1.3) Instruction preposition set PrepSet If the keyword k i The preceding word is identified as a preposition and recorded as the keyword preposition k i .prep, so the command preposition set comd.PrepSet = {<k1.prep,k1>,<k2.prep,k2>...,<k n .prep,k n >}, is a set of binary relationship groups for each keyword and its preposition; (1.4) Natural language instruction label set LabelSet The label set of a word w is represented as w.LabelSet = {l1,l2,...,l n }, then the natural language instruction label set is represented by the union of each keyword label set, comd.LabelSet = {k1.LabelSet, k2.LabelSet, ..., k n .LabelSet}={l1,l2,...,l m }; (2) API Features Define an API as API = {APIName, Instance, ParaName}, where APIName is the application interface name, Instance = {instance1, instance2, ..., instance n } are n natural language example instructions for calling the API, ParaName = {slot1, slot2, ..., slot m } is the m parameters required to execute the API. This parameter restriction is also called a word slot. The parameter value required to execute the API is the slot value. n .value; Similar to the natural language instruction features, the API features are defined as the structural features and semantic features of the example instructions for calling the API; therefore, the API features include three elements: API triple set api i .TripleSet, API label set api.KeySet, and API word slot label set api.LabelSet; (2.1) API Configuration Users can add x API configuration files as needed to meet their service scenarios; for each application program interface api i , users can personalize n service natural language examples Instance for it, and configure m word slots ParaName of the API; for each example instance j , configure the slot value corresponding to the API parameter in its natural language instruction n .value; (2.2) API triple set API triple collection api i .TripleSet contains a set of instruction triples for each example in the API. j .TripleSet; (2.3) API tag set Calculate all instances j The frequency of each label in .LabelSet i .f, if l i .f>F, then record l i is the core tag set of the API, and the calculation formula of F is as follows: F=w%×n API tag collection api i .LabelSet contains n examples of the service j All core tags of .KeySet; (2.4) API Slot Tag Set Use the general knowledge graph to interpret each set word slot of the API and obtain the API word slot label set; the API has m word slots that need to be filled. For multiple examples of an API, obtain their same word slot j Different slot values ​​under slot j .value tag set, after high-frequency processing of different slot value tag sets of the same word slot, the tag set api of the word slot is obtained i .ParaName.LabelSet; thereby obtaining the label set api for different word slots in the API i .slot j .LabelSet.

2. The method for constructing a personalized intelligent assistant based on a general knowledge graph according to claim 1, characterized in that: The implementation method of the API matching and recommendation based on similarity calculation is as follows: after obtaining the feature models of the user instructions and the APIs in multiple scenarios, the triple relationship similarity tri_sim and the label similarity lab_sim are used to perform similarity calculation to match the API; the triple relationship similarity tri_sim represents the action intention of two events, and by matching the relationship similarity, it is possible to determine whether the action intentions of the two events are similar; the label similarity lab_sim can determine whether the semantics of the two events are similar by calculating the label similarity; therefore, using the triple relationship similarity tri_sim and the label similarity lab_sim, the similarity API_Sim between the user instructions and the API in each scenario is obtained through weighted calculation, and the similarity result Top1 is taken as the matched API, and the scene matching result of the user instruction is returned.

3. The method for constructing a personalized intelligent assistant based on a general knowledge graph according to claim 2, characterized in that: The specific implementation process of the API matching and recommendation based on similarity calculation is as follows: (1) API similarity calculation based on triples The GloVe model trained with large-scale training corpus is used to represent the vocabulary in natural language text in a distributed manner. The sentence vector is represented by the average value of the sum of all predicate word vectors in the sentence. The sentence is converted into sentence predicate vectors with the same length for representation. The GloVe vector of each word w is represented as an m-dimensional vector vec(w)=(x i1 ,x i2 ,...,x im ); Define the sentence predicate vector V as the average of the word vectors of all relations with the same dimension, then the sentence is represented by an m-dimensional vector V, V = (V1, V2, ..., V m ), V j The calculation formula is as follows: If the natural language instruction input by the user is ucmd, then based on the word embedding technology, the user instruction triple set ucmd.TripleSet is vectorized; first, all triple instances of each instruction are traversed j .TripleSet, for each triple, take out each TripleSet.relation and add it to the triple relation set, ucmd.RelationSet = {relation1,relation2,...,relation m }; query the vector of each word in the Glove model; for a phrase, take the average to get the vector of the phrase; take the average of the vectors of all phrases to get the triple relationship vector TriVec of the entire instruction; the triple relationship vector of the user instruction is ucmd.TriVec=(utv1,utv2,...utv m ); Define API.relation as an API relation set, which contains the triple relations of all instructions in n examples of the API; instance j .RelationSet={r1,r2,...,r n };in, ri For the example instruction instance j .TripleSet each triple T i T i .relation; In the same way, based on the Glove model, we get the triple relation vector instance of each example j .TriVec, sum and average each example vector of the API to obtain the triple relationship api of each API i .TriVec=(atv1,atv2,...,atv m ); After finding the angle between the two sentence vectors, the cosine value is calculated to determine the similarity of the individuals; sentence a is represented by a vector (x1, x2, ..., x m ), sentence b is represented by a vector (y1,y2,...,y m ); then the similarity of two multidimensional vectors is calculated by the following formula: Use similarity calculation to compare the user instruction triple relationship vector ucmd.TriVec with the API triple relationship vector api i .TriVec uses the similarity calculation algorithm to obtain tri_sim, which is the similarity of the triple relationship between the user instruction and each API; the calculation formula is as follows: When the value of tri_sim is closer to 1, it means that the vector of the user input command is more similar to the vector of the API, which means that the action intention of the application scenario provided by the API is closest to the user action intention; (2) API similarity calculation based on tags (2.1) Label vectorization for APIs The label vectorization process refers to the process of vectorizing two label sets. Through vectorization, the similarity between the two label sets can be calculated, so as to determine the similarity between user instructions and APIs. For the label set, the bag-of-words model BOW is used to complete the vectorization. The bag-of-words model assumes that the word order, grammar and syntax of a text are ignored, and it is only regarded as a set of words or a combination of words. The appearance of each word in the text is independent and does not depend on whether other words appear. The user instruction needs to calculate the similarity with multiple APIs respectively to determine which API scenario the instruction is closest to. Therefore, the user instruction label set needs to be vectorized for each API to determine the overlap between the two label sets. Assume that each API label set is a vector consisting of only 1s; api i .LabelVec=(alv1,alv2,...,alv m )=(1,1,...,1) Input the feature models of all APIs and the feature models of user natural language instructions; according to the bag-of-words model, for each API, determine whether the words that appear in the user's natural language instruction tag set appear in the API tag set. If so, set it to 1, otherwise set it to 0; finally, for n APIs i Get n vectors containing 0 and 1 ucmd.LabelVec i ; Output the vector results of the user command for each API: ucmd.LabelVec i =(wolf1,wolf2,...,wolf m )wolf m ∈N∧wolf m ∈[0,1] (2.2) Tag-based API matching method By passing the user specified label vector ucmd.LabelVec i With all API tags vector api i .LabelVec performs similarity calculation to obtain the similarity lab_sim between the user instruction and the label set of each API. The calculation formula is as follows: (2.3) API recommendation similarity calculation The similarity between user instructions and APIs in each scenario, API_sim, takes into account two features: one is the structural triple feature, which can represent the action intention of an instruction through the relationship in the triple and distinguish different purposes; the other is the semantic label set feature, which adds label interpretations to the API through the general knowledge graph to represent the semantics of the API; therefore, API_sim is calculated by the triple relationship similarity tri_sim and the label set similarity lab_sim, and the formula is as follows, where α is the empirical value, α∈R + ∧α∈(0,1); API_sim=α×tri_sim+(1-α)×lab_sim The recommended similarity API_sim between the user command and each API is calculated by the above formula; the closer the value of API_sim is to 1, the closer the semantics expressed by the two are; therefore, according to the descending order, the API with the top 1 API similarity is obtained, and this API is returned as the final matched scenario.

4. The method for constructing a personalized intelligent assistant based on a general knowledge graph according to claim 1, characterized in that: The implementation method of the feature matching-based API parameter extraction method is as follows: using API word slot label similarity KeyLabel_sim and preposition similarity KeyPrep_sim to extract and match keywords, and obtaining the word slot similarity Key_Sim through weighted calculation; sorting by similarity of the word slot similarity Key_Sim, and obtaining the parameters required for API execution from user instructions.

5. The method for constructing a personalized intelligent assistant based on a general knowledge graph according to claim 4, characterized in that: The specific implementation process of the API parameter extraction method based on feature matching is as follows: (1) For each keyword label vector of the user instruction and each word slot label vector of the API, the label similarity KeyLabel_sim is calculated using cosine similarity. The calculation formula is as follows: (2) Calculate the similarity between prepositions KeyPrep_sim Use the Glove model to vectorize each preposition word and vectorize each keyword k in the user instruction. i The corresponding preposition k1.prep is taken from the preposition set and vectorized based on the Glove model as k i .PrepVec=(upv1,upv2,...,upv m ); The prefix of the API slot value is the prefix set of the API example instruction; Find the slot value in the API example j .value, extract the word slot preposition value.prep, the vectorized representation result of the API preposition is slot j .value.PrepVec=(apv1,apv2,...,apv m ); Use cosine similarity to calculate the similarity between the keyword preposition in the user command and the API word slot preposition vector KeyPrep_sim; The calculation formula is as follows: (3) Calculate the similarity between the user command keywords and the word slots Use weight calculation to get the word slot similarity Key_sim; Calculate which keyword in the user instruction has the closest meaning to the word slot, so as to find the closest keyword as the slot value of the word slot and fill it into the API; the formula is as follows, where β is the empirical value, β∈R + ∧β∈(0,1); Key_sim=β×KeyLabel_sim+(1-β)×KeyPrep_sim Sorting is performed in descending order according to the word slot similarity, and the slot value corresponding to a word slot in the user instruction is obtained from the high to low similarity, that is, the parameters required to execute the API are extracted from the user instruction, and a word slot matching list is obtained, that is, the parameters required to execute the API are obtained to complete the API execution.

Images