Multimodal interaction method and system for intelligent question-asking platform

Abstract

The present invention discloses a multimodal interaction method and system for an intelligent question-and-answer platform, and relates to the technical field of multimodal interaction. The interaction method comprises the following steps: arbitrarily selecting an interaction record, analyzing the interaction information stored in the interaction record, establishing different storage nodes and sequentially connecting them; dividing all storage nodes according to the interaction information stored in each storage node, generating several types of storage sets; analyzing the connection relationship between each type of storage set, and obtaining the degree of association between any two types of storage sets; arbitrarily selecting two storage nodes, and analyzing the connection relationship between the two storage nodes; respectively obtaining the two types of storage sets and analyzing the degree of association between the two types of storage sets, and calculating the association value between the two storage nodes; analyzing the association value between a certain storage node and other storage nodes, obtaining the recommended value of the certain storage node, and pushing information to users.

Description

Technical Field

[0001] The present invention relates to the field of multimodal interaction technology, and in particular to a multimodal interaction method and system for an intelligent question-asking platform. Background Art

[0002] The Intelligent Question and Answer Platform uses artificial intelligence, especially large language models, to extract data insights and help users conduct fast and effective data query and analysis. Multimodal interaction means that the system can simultaneously process and understand multiple forms of information input, such as text, voice, images, videos, etc. This interactive method allows users to access and understand data in a more natural and diverse way.

[0003] In today's era of information explosion, users are faced with a large amount of information and content choices; users often get lost in the massive amount of content and are unable to quickly find the content they are interested in, which increases the user's choice burden; although the Smart Question and Answer Platform can help users filter relevant content based on user needs; however, when users are not clear about their needs, the platform cannot effectively help users filter out information of interest. Summary of the invention

[0004] The purpose of the present invention is to provide a multimodal interaction method and system for an intelligent question-and-answer platform to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solution: a multimodal interaction method for an intelligent question-asking platform, the interaction method comprising the following steps:

[0006] Step S100: construct a multimodal interaction system, store each interaction process of the user on the intelligent Q&A platform, and generate an interaction record; select an interaction record at random, analyze the correlation between the interaction information stored in the interaction record and the difference in the interaction mode, and establish different storage nodes for the interaction information in the interaction record;

[0007] Step S200: sequentially connect the storage nodes in each interaction record; obtain the storage nodes of all interaction records, divide all storage nodes according to the interaction information stored in each storage node, and generate several types of storage sets; analyze the connection relationship between the various types of storage sets to obtain the degree of association between any two types of storage sets;

[0008] Step S300: arbitrarily selecting two storage nodes, analyzing the connection relationship between the two storage nodes in each interaction record; respectively obtaining two types of storage sets where the two storage nodes are located, analyzing the degree of association between the two types of storage sets, and calculating the association value between the two storage nodes;

[0009] Step S400: arbitrarily select a storage node, analyze the association value between the storage node and other storage nodes, and obtain the recommended value of the storage node; obtain the recommended value of each storage node, generate the recommended priority of the interactive information stored in each storage node, and push the information to the user.

[0010] Furthermore, step S100 includes the following steps:

[0011] Step S101: arbitrarily obtain the interaction information stored in the ith interaction record, extract several pieces of information generated by the intelligent question-asking platform from the interaction information; arbitrarily select the jth piece of information from the several pieces of information, perform word segmentation processing on the jth piece of information and extract several feature words, and generate a feature set R of the jth piece of information j ;

[0012] Step S102: capturing the interaction mode between the user and the intelligent question-asking platform during the process of the intelligent question-asking platform generating the j-th piece of information, and obtaining the interaction form H of the j-th piece of information j ; Generate the information set M of the jth information j =(R j ,H j );

[0013] Step S103: arbitrarily obtain the feature sets of two adjacent information in the i-th interaction record, arbitrarily extract a feature word from each of the two feature sets, perform similarity calculation on the two extracted feature words, set a similarity threshold, and if the calculated similarity exceeds the similarity threshold, set the two feature words as related words; count the number of related words in the two feature sets as W k , set the total number of feature words contained in the two feature sets to W all , the similarity of the two feature sets is calculated to be η=W k / W all ; In one interaction record, there are multiple conversations between the user and the smart question platform, and there will be connections between the information in the conversation. Therefore, it is necessary to compare the similarity of the information in one interaction record and divide it. The number of times the user queries each type of information can be obtained, which is conducive to the setting of subsequent message priority push;

[0014] Step S104: Set the similarity threshold to n max , if η max <η, then two storage nodes are established to store the two pieces of information respectively. If η max≥η, the two pieces of information are stored in the same storage node, and several storage nodes of the i-th interaction record are obtained; if the similarity of the two pieces of information is high, it indicates that they are of the same type, and only one storage node is needed for storage; while the low similarity indicates that the user queried two types of information, and two storage nodes are needed for storage, which is conducive to the subsequent judgment of the user's preference information according to the division of the storage nodes.

[0015] Further, step S200 includes the following steps:

[0016] Step S201: arbitrarily obtain the i-th interaction record, and obtain the information generation time of each piece of information in the i-th interaction record; extract the information stored in each storage node in the i-th interaction record, and establish a connection relationship between the storage nodes according to the order of the information generation time of each piece of information; according to the interaction order between the user and the intelligent question-and-answer platform in an interaction record, the storage nodes are connected, which can directly reflect the correlation between various types of information;

[0017] Step S202: Obtain all storage nodes established in each interaction record and the connection relationship between each storage node; extract one piece of information from each storage node to generate a feature information set, select two pieces of feature information from the feature information set and extract the feature sets of the two pieces of feature information respectively, calculate the similarity of the two feature sets, and if the similarity is greater than a set similarity threshold, merge the storage nodes where the two pieces of feature information are located to generate a type of storage set; divide all storage nodes to generate several types of storage sets;

[0018] Step S203: arbitrarily select the k1-th storage set and the k2-th storage set, and obtain the number of storage nodes in the k1-th storage set as N. k1 and the number of storage nodes in the k2-th storage set is N k2 ; Obtain the connection relationship between each storage node in the two types of storage sets, and count the number of connections between each storage node in the two types of storage sets as N k1,k2 ; According to the formula:

[0019]

[0020] Calculate the correlation degree G between the k1 storage set and the k2 storage set k1,k2 ; According to the connection relationship between each storage node in the two types of storage sets, the degree of association between the two storage sets can be most directly reflected.

[0021] Further, step S300 includes the following steps:

[0022] Step S301: setting the two selected storage nodes to be the a1th storage node and the a2th storage node, obtaining that the a1th storage node is stored in the k1th storage set and the a2th storage node is stored in the k2th storage set;

[0023] Step S302: Obtain the association degree G between the k1-th storage set and the k2-th storage set k1,k2 , get the number of connections between the two types of storage sets, the number of connections between the a1th storage node and the a2th storage node is M a1,a2 , the direct correlation degree between the a1th storage node and the a2th storage node is obtained as P a1,a2 =G k1,k2 ×(M a1,a2 / N k1,k2 ), where N k1,k2 is the number of connections between each storage node between the k1-th storage set and the k2-th storage set;

[0024] Step S303: extract a storage node from the k1-th storage set and the k2-th storage set at random, extract the connection line between the two storage nodes from each interaction record, and generate a connection route set of the two storage nodes; select the b-th connection route from the connection route set, obtain the degree of association between the storage sets where any two adjacent connection nodes in the b-th connection route are located, and accumulate all the obtained degrees of association to calculate the indirect degree of association Q between the two storage nodes in the b-th connection route. b ;

[0025] Step S304: Obtain the indirect association degree between the two storage nodes in each connection route of the connection route set; according to the formula:

[0026]

[0027] Wherein, x is the number of connection routes in the connection route set; the association value Z between the a1th storage node and the a2th storage node is calculated. a1,a2 ;

[0028] The association value between two storage nodes is mainly obtained from two parts. One part is the association between the two storage nodes, and the other part is the association between the two storage nodes generated through the transformation between other storage nodes. Therefore, the association between the two parts needs to be considered comprehensively to obtain an accurate association value.

[0029] Furthermore, step S400 includes the following steps:

[0030] Step S401: obtaining the association value between a selected storage node and the remaining storage nodes, and calculating the total association value of the storage node to be Z; obtaining the number of the remaining storage nodes to be Y, and calculating the recommended value of the storage node to be T=Y×Z; the recommended value of a storage node is affected by the number of associated storage nodes and the association value;

[0031] Step S402: sorting all storage nodes from large to small according to the numerical value of the recommended value, generating a recommended priority of the information stored in each storage node; obtaining an information set of each piece of information in each storage node, counting the number of various interaction modes in each information set, and selecting an interaction mode with the largest number as the target interaction mode for each piece of information;

[0032] Step S403: in the multimodal interaction system, information is pushed to the user according to the recommendation priority, wherein the method of pushing information is the target interaction method.

[0033] In order to better implement the above method, a multimodal interactive system for the intelligent question-asking platform is also proposed. The interactive system includes an interactive record analysis module, a storage set analysis module, a storage node association module and an interactive information push module.

[0034] The interaction record analysis module is used to build a multimodal interaction system, store each interaction process of the user on the intelligent Q&A platform, and generate an interaction record; select an interaction record at random, analyze the correlation between the interaction information stored in the interaction record and the difference in the interaction mode, and establish different storage nodes for the interaction information in the interaction record;

[0035] The storage set analysis module is used to sequentially connect the storage nodes in each interaction record; obtain the storage nodes of all interaction records, divide all storage nodes according to the interaction information stored in each storage node, and generate several types of storage sets; analyze the connection relationship between various types of storage sets, and obtain the degree of association between any two types of storage sets;

[0036] The storage node association module is used to select two storage nodes at random, analyze the connection relationship between the two storage nodes in each interaction record; respectively obtain two types of storage sets where the two storage nodes are located, analyze the degree of association between the two types of storage sets, and calculate the association value between the two storage nodes;

[0037] The interactive information push module is used to arbitrarily select a storage node, analyze the association value between the storage node and other storage nodes, and obtain the recommended value of the storage node; obtain the recommended value of each storage node, generate the recommended priority of the interactive information stored in each storage node, and push the information to the user.

[0038] Further, the interaction record analysis module includes an interaction record generation unit and a storage node establishment unit;

[0039] The interaction record generation unit is used to build a multimodal interaction system, store each interaction process of the user on the intelligent Q&A platform, and generate an interaction record; the storage node establishment unit is used to arbitrarily select an interaction record, analyze the correlation between the interaction information stored in the interaction record and the difference in the interaction mode, and establish different storage nodes for the interaction information in the interaction record.

[0040] Further, the storage set analysis module includes a storage set partitioning unit and a correlation degree analysis unit;

[0041] The storage set division unit is used to sequentially connect the storage nodes in each interaction record; obtain the storage nodes of all interaction records, divide all storage nodes according to the interaction information stored in each storage node, and generate several types of storage sets; the association degree analysis unit is used to analyze the connection relationship between various types of storage sets and obtain the association degree between any two types of storage sets.

[0042] Further, the storage node association module includes a node connection analysis unit and an association value calculation unit;

[0043] The node connection analysis unit is used to arbitrarily select two storage nodes and analyze the connection relationship between the two storage nodes in each interaction record; the association value calculation unit is used to respectively obtain the two types of storage sets where the two storage nodes are located, analyze the degree of association between the two types of storage sets, and calculate the association value between the two storage nodes.

[0044] Further, the interactive information push module includes a recommendation value calculation unit and an information push setting unit;

[0045] The recommendation value calculation unit is used to arbitrarily select a storage node, analyze the association value between the storage node and other storage nodes, and obtain the recommendation value of the storage node; the information push setting unit is used to obtain the recommendation value of each storage node, generate the recommendation priority of the interactive information stored in each storage node, and push the information to the user.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] 1. The present invention analyzes the user's historical interactive information to filter out the information that the user is interested in; when the user is not clear about the needs, it can automatically help the user filter out the information of interest, help the user select information, and improve the efficiency of the user's information access;

[0048] 2. The present invention divides the user's interactive information and generates storage nodes to obtain the number of various types of information that the user has historically consulted, which can better help the user analyze the information of interest;

[0049] 3. The present invention obtains the correlation between each storage node by analyzing the association between each storage node and each type of storage collection; based on the correlation between each storage node, it can reflect the probability of the platform pushing various types of information when the user historically searches for information, and can help the user analyze the information of interest, and automatically select messages to help the user search for information. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the steps of a multimodal interaction method for an intelligent question-asking platform;

[0051] Figure 2 Schematic diagram of the structure of the multimodal interaction system for the intelligent question-and-answer platform. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Example: Figure 1 to Figure 2 As shown, the present invention provides a multimodal interaction method for an intelligent question-asking platform, and the interaction method comprises the following steps:

[0054] Step S100: construct a multimodal interaction system, store each interaction process of the user on the intelligent Q&A platform, and generate an interaction record; select an interaction record at random, analyze the correlation between the interaction information stored in the interaction record and the difference in the interaction mode, and establish different storage nodes for the interaction information in the interaction record;

[0055] Wherein, step S100 includes the following steps:

[0056] Step S101: arbitrarily obtain the interaction information stored in the ith interaction record, extract several pieces of information generated by the intelligent question-asking platform from the interaction information; arbitrarily select the jth piece of information from the several pieces of information, perform word segmentation processing on the jth piece of information and extract several feature words, and generate a feature set R of the jth piece of information j ;

[0057] Step S102: capturing the interaction mode between the user and the intelligent question-asking platform during the process of the intelligent question-asking platform generating the j-th piece of information, and obtaining the interaction form H of the j-th piece of information j ; Generate the information set M of the jth information j =(R j ,H j );

[0058] Step S103: arbitrarily obtain the feature sets of two adjacent information in the i-th interaction record, arbitrarily extract a feature word from each of the two feature sets, perform similarity calculation on the two extracted feature words, set a similarity threshold, and if the calculated similarity exceeds the similarity threshold, set the two feature words as related words; count the number of related words in the two feature sets as W k , set the total number of feature words contained in the two feature sets to W all , the similarity of the two feature sets is calculated to be η=W k / W all ;

[0059] Step S104: Set the similarity threshold to n max , if η max <η, then two storage nodes are established to store the two pieces of information respectively. If η max ≥η, the two pieces of information are stored in the same storage node to obtain several storage nodes for the i-th interaction record.

[0060] Step S200: sequentially connect the storage nodes in each interaction record; obtain the storage nodes of all interaction records, divide all storage nodes according to the interaction information stored in each storage node, and generate several types of storage sets; analyze the connection relationship between the various types of storage sets to obtain the degree of association between any two types of storage sets;

[0061] Wherein, step S200 includes the following steps:

[0062] Step S201: arbitrarily obtain the i-th interaction record, and obtain the information generation time of each piece of information in the i-th interaction record; extract the information stored in each storage node in the i-th interaction record, and establish a connection relationship between the storage nodes in the order of the information generation time of each piece of information;

[0063] Embodiment 1: According to the information generation time of each piece of information in the interaction record, the information generation order is obtained as A, B, C, so the connection relationship between the storage nodes is A connected to B, and B connected to C;

[0064] Step S202: Obtain all storage nodes established in each interaction record and the connection relationship between each storage node; extract one piece of information from each storage node to generate a feature information set, select two pieces of feature information from the feature information set and extract the feature sets of the two pieces of feature information respectively, calculate the similarity of the two feature sets, and if the similarity is greater than a set similarity threshold, merge the storage nodes where the two pieces of feature information are located to generate a type of storage set; divide all storage nodes to generate several types of storage sets;

[0065] Step S203: arbitrarily select the k1-th storage set and the k2-th storage set, and obtain the number of storage nodes in the k1-th storage set as N. k1 and the number of storage nodes in the k2-th storage set is N k2 ; Obtain the connection relationship between each storage node in the two types of storage sets, and count the number of connections between each storage node in the two types of storage sets as N k1,k2 ; According to the formula:

[0066]

[0067] Calculate the correlation degree G between the k1 storage set and the k2 storage set k1,k2 ;

[0068] Embodiment 2: Set the number of storage nodes in the k1 storage set to 10, and the number of storage nodes in the k2 storage set to 10; extract one storage node from each of the two storage sets, and if there is a connection relationship between the two storage nodes, the number of connections is increased by 1; finally, the number of connections between the storage nodes in the two storage sets is 10, and the degree of association is G=10 / 20=50%;

[0069] Step S300: arbitrarily selecting two storage nodes, analyzing the connection relationship between the two storage nodes in each interaction record; respectively obtaining two types of storage sets where the two storage nodes are located, analyzing the degree of association between the two types of storage sets, and calculating the association value between the two storage nodes;

[0070] Wherein, step S300 includes the following steps:

[0071] Step S301: setting the two selected storage nodes to be the a1th storage node and the a2th storage node, obtaining that the a1th storage node is stored in the k1th storage set and the a2th storage node is stored in the k2th storage set;

[0072] Step S302: Obtain the association degree G between the k1-th storage set and the k2-th storage set k1,k2, get the number of connections between the two types of storage sets, the number of connections between the a1th storage node and the a2th storage node is M a1,a2 , the direct correlation degree between the a1th storage node and the a2th storage node is obtained as P a1,a2 =G k1,k2 ×(M a1,a2 / N k1,k2 ), where N k1,k2 is the number of connections between each storage node between the k1-th storage set and the k2-th storage set;

[0073] Example 3: The association degree between the two types of storage sets is set to 50%, and the number of connections between the two types of storage sets is 10 times, but the number of connections between the a1 storage node and the a2 storage node is 2 times, so the direct association degree between the a1 storage node and the a2 storage node is P = 50% × (2 / 10) = 10%;

[0074] Step S303: extract a storage node from the k1-th storage set and the k2-th storage set at random, extract the connection line between the two storage nodes from each interaction record, and generate a connection route set of the two storage nodes; select the b-th connection route from the connection route set, obtain the degree of association between the storage sets where any two adjacent connection nodes in the b-th connection route are located, and accumulate all the obtained degrees of association to calculate the indirect degree of association Q between the two storage nodes in the b-th connection route. b ;

[0075] Step S304: Obtain the indirect association degree between the two storage nodes in each connection route of the connection route set; according to the formula:

[0076]

[0077] Wherein, x is the number of connection routes in the connection route set; the association value Z between the a1th storage node and the a2th storage node is calculated. a1,a2 ;

[0078] Example 4: Assume that there are 3 connection routes between two storage nodes, and the indirect correlation degrees are 5%, 7% and 8% respectively; set the direct correlation degree to P=10%, so the correlation value is Z=10%+20%=30%.

[0079] Step S400: arbitrarily select a storage node, analyze the association value between the storage node and other storage nodes, and obtain the recommended value of the storage node; obtain the recommended value of each storage node, generate the recommended priority of the interactive information stored in each storage node, and push the information to the user;

[0080] Wherein, step S400 includes the following steps:

[0081] Step S401: obtaining the association value between a selected storage node and the remaining storage nodes, and calculating the total association value of the storage node as Z; obtaining the number of the remaining storage nodes as Y, and calculating the recommended value of the storage node as T=Y×Z;

[0082] Step S402: sorting all storage nodes from large to small according to the numerical value of the recommended value, generating a recommended priority of the information stored in each storage node; obtaining an information set of each piece of information in each storage node, counting the number of various interaction modes in each information set, and selecting an interaction mode with the largest number as the target interaction mode for each piece of information;

[0083] Step S403: in the multimodal interaction system, information is pushed to the user according to the recommendation priority, wherein the method of pushing information is the target interaction method.

[0084] A multimodal interactive system for the intelligent question-asking platform, the interactive system includes an interactive record analysis module, a storage set analysis module, a storage node association module and an interactive information push module;

[0085] The interaction record analysis module is used to build a multimodal interaction system, store each interaction process of the user on the intelligent Q&A platform, and generate an interaction record; select an interaction record at random, analyze the correlation between the interaction information stored in the interaction record and the difference in the interaction mode, and establish different storage nodes for the interaction information in the interaction record;

[0086] The storage set analysis module is used to sequentially connect the storage nodes in each interaction record; obtain the storage nodes of all interaction records, divide all storage nodes according to the interaction information stored in each storage node, and generate several types of storage sets; analyze the connection relationship between various types of storage sets, and obtain the degree of association between any two types of storage sets;

[0087] The storage node association module is used to select two storage nodes at random, analyze the connection relationship between the two storage nodes in each interaction record; respectively obtain two types of storage sets where the two storage nodes are located, analyze the degree of association between the two types of storage sets, and calculate the association value between the two storage nodes;

[0088] The interactive information push module is used to arbitrarily select a storage node, analyze the association value between the storage node and other storage nodes, and obtain the recommended value of the storage node; obtain the recommended value of each storage node, generate the recommended priority of the interactive information stored in each storage node, and push the information to the user.

[0089] Among them, the interaction record analysis module includes an interaction record generation unit and a storage node establishment unit;

[0090] The interaction record generation unit is used to build a multimodal interaction system, store each interaction process of the user on the intelligent Q&A platform, and generate an interaction record; the storage node establishment unit is used to arbitrarily select an interaction record, analyze the correlation between the interaction information stored in the interaction record and the difference in the interaction mode, and establish different storage nodes for the interaction information in the interaction record.

[0091] Wherein, the storage set analysis module includes a storage set partitioning unit and a correlation degree analysis unit;

[0092] The storage set division unit is used to sequentially connect the storage nodes in each interaction record; obtain the storage nodes of all interaction records, divide all storage nodes according to the interaction information stored in each storage node, and generate several types of storage sets; the association degree analysis unit is used to analyze the connection relationship between various types of storage sets and obtain the association degree between any two types of storage sets.

[0093] Wherein, the storage node association module includes a node connection analysis unit and an association value calculation unit;

[0094] The node connection analysis unit is used to arbitrarily select two storage nodes and analyze the connection relationship between the two storage nodes in each interaction record; the association value calculation unit is used to respectively obtain the two types of storage sets where the two storage nodes are located, analyze the degree of association between the two types of storage sets, and calculate the association value between the two storage nodes.

[0095] Among them, the interactive information push module includes a recommendation value calculation unit and an information push setting unit;

[0096] The recommendation value calculation unit is used to arbitrarily select a storage node, analyze the association value between the storage node and other storage nodes, and obtain the recommendation value of the storage node; the information push setting unit is used to obtain the recommendation value of each storage node, generate the recommendation priority of the interactive information stored in each storage node, and push the information to the user.

[0097] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A multimodal interaction method for an intelligent question-asking platform, characterized in that: The interactive method comprises the following steps: Step S100: construct a multimodal interaction system, store each interaction process of the user on the intelligent Q&A platform, and generate an interaction record; select an interaction record at random, analyze the correlation between the interaction information stored in the interaction record and the difference in the interaction mode, and establish different storage nodes for the interaction information in the interaction record; Step S200: sequentially connect the storage nodes in each interaction record; obtain the storage nodes of all interaction records, divide all storage nodes according to the interaction information stored in each storage node, and generate several types of storage sets; analyze the connection relationship between the various types of storage sets to obtain the degree of association between any two types of storage sets; Step S300: arbitrarily selecting two storage nodes, analyzing the connection relationship between the two storage nodes in each interaction record; respectively obtaining two types of storage sets where the two storage nodes are located, analyzing the degree of association between the two types of storage sets, and calculating the association value between the two storage nodes; Step S400: arbitrarily select a storage node, analyze the association value between the storage node and other storage nodes, and obtain the recommended value of the storage node; obtain the recommended value of each storage node, generate the recommended priority of the interactive information stored in each storage node, and push the information to the user; The step S100 includes the following steps: Step S101: arbitrarily obtain the interaction information stored in the ith interaction record, extract several pieces of information generated by the intelligent question-asking platform from the interaction information; arbitrarily select the jth piece of information from the several pieces of information, perform word segmentation processing on the jth piece of information and extract several feature words, and generate a feature set R of the jth piece of information j ; Step S102: capturing the interaction mode between the user and the intelligent question-asking platform during the process of the intelligent question-asking platform generating the j-th piece of information, and obtaining the interaction form H of the j-th piece of information j ; Generate the information set M of the jth information j =(R j ,H j ); Step S103: arbitrarily obtain the feature sets of two adjacent information in the i-th interaction record, arbitrarily extract a feature word from each of the two feature sets, perform similarity calculation on the two extracted feature words, set a similarity threshold, and if the calculated similarity exceeds the similarity threshold, set the two feature words as related words; count the number of related words in the two feature sets as W k , set the total number of feature words contained in the two feature sets to W all , the similarity of the two feature sets is calculated to be η = W k / W all ; Step S104: Set the similarity threshold to n max , if η max <η, then two storage nodes are established to store the two pieces of information respectively. If η max ≥η, the two pieces of information are stored in the same storage node to obtain several storage nodes for the i-th interaction record; The step S200 includes the following steps: Step S201: arbitrarily obtain the i-th interaction record, and obtain the information generation time of each piece of information in the i-th interaction record; extract the information stored in each storage node in the i-th interaction record, and establish a connection relationship between the storage nodes in the order of the information generation time of each piece of information; Step S202: Obtain all storage nodes established in each interaction record and the connection relationship between each storage node; extract one piece of information from each storage node to generate a feature information set, select two pieces of feature information from the feature information set and extract the feature sets of the two pieces of feature information respectively, calculate the similarity of the two feature sets, and if the similarity is greater than a set similarity threshold, merge the storage nodes where the two pieces of feature information are located to generate a type of storage set; divide all storage nodes to generate several types of storage sets; Step S203: arbitrarily select the k1-th storage set and the k2-th storage set, and obtain the number of storage nodes in the k1-th storage set as N. k1 and the number of storage nodes in the k2-th storage set is N k2 ; Obtain the connection relationship between each storage node in the two types of storage sets, and count the number of connections between each storage node in the two types of storage sets as N k1,k2 ; According to the formula: Calculate the correlation degree G between the k1 storage set and the k2 storage set k1,k2 .

2. The multimodal interaction method for the intelligent question-asking platform according to claim 1, characterized in that: The step S300 includes the following steps: Step S301: setting the two selected storage nodes to be the a1th storage node and the a2th storage node, obtaining that the a1th storage node is stored in the k1th storage set and the a2th storage node is stored in the k2th storage set; Step S302: Obtain the association degree G between the k1-th storage set and the k2-th storage set k1,k2 , get the number of connections between the two types of storage sets, the number of connections between the a1th storage node and the a2th storage node is M a1,a2 , the direct correlation degree between the a1th storage node and the a2th storage node is obtained as P a1,a2 =G k1,k2 ×(M a1,a2 / N k1,k2 ), where N k1,k2 is the number of connections between each storage node between the k1-th storage set and the k2-th storage set; Step S303: extract a storage node from the k1-th storage set and the k2-th storage set at random, extract the connection line between the two storage nodes from each interaction record, and generate a connection route set of the two storage nodes; select the b-th connection route from the connection route set, obtain the degree of association between the storage sets where any two adjacent connection nodes in the b-th connection route are located, and accumulate all the obtained degrees of association to calculate the indirect degree of association Q between the two storage nodes in the b-th connection route. b ; Step S304: Obtain the indirect association degree between the two storage nodes in each connection route of the connection route set; according to the formula: Wherein, x is the number of connection routes in the connection route set; the association value Z between the a1th storage node and the a2th storage node is calculated. a1,a2 .

3. The multimodal interaction method for the intelligent question-asking platform according to claim 2, characterized in that: The step S400 includes the following steps: Step S401: obtaining the association value between a selected storage node and the remaining storage nodes, and calculating the total association value of the storage node as Z; obtaining the number of the remaining storage nodes as Y, and calculating the recommended value of the storage node as T=Y×Z; Step S402: sorting all storage nodes from large to small according to the numerical value of the recommended value, generating a recommended priority of the information stored in each storage node; obtaining an information set of each piece of information in each storage node, counting the number of various interaction modes in each information set, and selecting an interaction mode with the largest number as the target interaction mode for each piece of information; Step S403: in the multimodal interaction system, information is pushed to the user according to the recommendation priority, wherein the method of pushing information is the target interaction method.

4. A multimodal interaction system, for executing the multimodal interaction method for an intelligent question-asking platform as claimed in any one of claims 1 to 3, characterized in that: The interactive system includes an interactive record analysis module, a storage set analysis module, a storage node association module and an interactive information push module; The interaction record analysis module is used to build a multimodal interaction system, store each interaction process of the user on the intelligent question platform, and generate an interaction record; randomly select an interaction record, analyze the correlation between the interaction information stored in the interaction record and the difference in the interaction mode, and establish different storage nodes for the interaction information in the interaction record; The storage set analysis module is used to sequentially connect the storage nodes in each interaction record; Obtain the storage nodes of all interaction records, divide all storage nodes according to the interaction information stored in each storage node, and generate several types of storage sets; analyze the connection relationship between each type of storage set, and obtain the degree of association between any two types of storage sets; The storage node association module is used to select two storage nodes at random, analyze the connection relationship between the two storage nodes in each interaction record; respectively obtain two types of storage sets where the two storage nodes are located, analyze the degree of association between the two types of storage sets, and calculate the association value between the two storage nodes; The interactive information push module is used to arbitrarily select a storage node, analyze the association value between the storage node and other storage nodes, and obtain the recommended value of the storage node; obtain the recommended value of each storage node, generate the recommended priority of the interactive information stored in each storage node, and push the information to the user.

5. The multimodal interaction system according to claim 4, characterized in that: The interaction record analysis module includes an interaction record generation unit and a storage node establishment unit; The interaction record generation unit is used to build a multimodal interaction system, store each interaction process of the user on the intelligent question platform, and generate an interaction record; The storage node establishing unit is used to select an interaction record at random, analyze the relevance between the interaction information stored in the interaction record and the difference in interaction modes, and establish different storage nodes for the interaction information in the interaction record.

6. The multimodal interaction system according to claim 4, characterized in that: The storage set analysis module includes a storage set partitioning unit and a correlation degree analysis unit; The storage set division unit is used to sequentially connect the storage nodes in each interaction record; obtain the storage nodes of all interaction records, divide all storage nodes according to the interaction information stored in each storage node, and generate several types of storage sets; The association degree analysis unit is used to analyze the connection relationship between various types of storage sets to obtain the association degree between any two types of storage sets.

7. The multimodal interaction system according to claim 4, characterized in that: The storage node association module includes a node connection analysis unit and an association value calculation unit; The node connection analysis unit is used to select any two storage nodes and analyze the connection relationship between the two storage nodes in each interaction record; The association value calculation unit is used to respectively obtain the two types of storage sets where the two storage nodes are located, analyze the degree of association between the two types of storage sets, and calculate the association value between the two storage nodes.

8. The multimodal interaction system according to claim 4, characterized in that: The interactive information push module includes a recommendation value calculation unit and an information push setting unit; The recommended value calculation unit is used to arbitrarily select a storage node, analyze the association value between the storage node and other storage nodes, and obtain the recommended value of the storage node; the information push setting unit is used to obtain the recommended value of each storage node, generate the recommended priority of the interactive information stored in each storage node, and push information to the user.

Images