Intelligent dialogue method and device, electronic device, and storage medium

Abstract

The application provides an intelligent conversation method and device, an electronic device and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: acquiring position information and environmental audio information of a user; determining a space scene classification in which the user is located based on the position information and the environmental audio information; in response to receiving a conversation instruction of the user, determining a conversation topic based on the space scene classification and the conversation instruction, and performing intelligent conversation based on the conversation topic. The intelligent conversation method and device, the electronic device and the storage medium provided by the application can generate reply information that conforms to the real interests of the user, and improve the user experience.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and more specifically, relates to an intelligent dialogue method and device, electronic device, and storage medium. Background Technology

[0002] With the development of natural language processing technology, intelligent chat tools can recognize and understand the language input by users and generate appropriate responses, enabling smooth dialogue between humans and machines. Applying intelligent chat technology to wearable devices can meet users' needs for convenient communication and information access in their daily lives, giving wearable devices more functions and value, making them a powerful assistant in users' lives.

[0003] However, existing intelligent dialogue methods mostly rely on surface-level text matching to understand user intent, resulting in responses that fail to match users' true interests and negatively impact user experience. Summary of the Invention

[0004] The purpose of this application is to provide an intelligent dialogue method, device, electronic device, and storage medium to generate response information that matches the user's real interests and improve the user experience.

[0005] A first aspect of this application provides an intelligent dialogue method, including:

[0006] Obtain the user's location information and ambient audio information;

[0007] Based on the location information and environmental audio information, the spatial scene classification of the user is determined;

[0008] In response to receiving a user's dialogue command, the system determines a dialogue topic based on the spatial scene classification and the dialogue command, and then conducts intelligent dialogue based on the dialogue topic.

[0009] A second aspect of this application provides an intelligent dialogue device, comprising:

[0010] The data acquisition module is used to acquire the user's location information and environmental audio information;

[0011] The scene classification module is used to determine the spatial scene classification of the user based on the location information and environmental audio information;

[0012] The intelligent dialogue module is used to respond to a user's dialogue command, determine the dialogue topic based on the spatial scene classification and the dialogue command, and conduct intelligent dialogue based on the dialogue topic.

[0013] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the intelligent dialogue method described above.

[0014] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described intelligent dialogue method.

[0015] The beneficial effects of the intelligent dialogue method, apparatus, electronic device, and storage medium provided in this application are as follows:

[0016] This application embodiment takes into account that users are interested in different topics under different spatial scene categories. Based on the user's location information and environmental audio information, the spatial scene category in which the user is located is determined. When a user command is received, the dialogue topic is determined by combining the spatial scene category and the user's dialogue command. This can make the dialogue topic more in line with the user's interests, thereby improving the user experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an intelligent dialogue method provided in an embodiment of this application;

[0019] Figure 2 This is a structural block diagram of an intelligent dialogue device provided in an embodiment of this application;

[0020] Figure 3 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] It is understood that in the embodiments of this application, data related to user information (such as location information) is involved. When the embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with relevant laws, regulations and standards.

[0023] It should be noted that the terms "first," "second," etc., used in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein.

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0025] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the intelligent dialogue method provided in this application. The intelligent dialogue method provided in this application embodiment can be executed by an electronic device, and the method may include:

[0026] S101: Obtain the user's location information and ambient audio information.

[0027] In this embodiment, the user's location information can be obtained using the built-in Global Positioning System (GPS positioning module) of the electronic device, and environmental audio information can be collected using the built-in microphone array of the electronic device. The collection of the user's location information and environmental audio information can be triggered according to set trigger conditions. These trigger conditions can be timed triggering or triggering each time the electronic device is woken up. When the electronic device is in a woken-up state, the corresponding time interval can be set to be shorter (e.g., 10 minutes), and when the electronic device is in a standby state, the corresponding time interval can be set to be longer (e.g., 1 hour). Alternatively, the collection of the user's location information and environmental audio information can be set to be triggered only during specific time periods (e.g., 7:00 AM to 9:00 PM daily), and not during other time periods, to reduce unnecessary power consumption.

[0028] S102: Determine the spatial scene classification of the user based on location information and environmental audio information.

[0029] In this embodiment, the spatial scene classification of the user may include commuting scenarios, outdoor leisure scenarios, indoor static scenarios, and sports and fitness scenarios. Under different spatial scene classifications, the corresponding location information and environmental audio information are significantly different.

[0030] For example, in commuting scenarios, the movement speed is stable (e.g., 60-80 km / h for subways, 30-50 km / h for buses), and Points of Interest (POIs) are mainly transportation hubs (subway stations, bus stops, main roads). The main sound sources are traffic noise and background noise from people, such as the friction of subway tracks and station announcements, and the engine and arrival announcements of buses, mixed with passenger conversations. In outdoor leisure scenarios, the movement speed is usually <5 km / h (e.g., walking, picnicking), and POIs are mainly leisure places (parks, squares, scenic spots, or campsites). The main sound sources are natural sounds and sounds of leisure activities, such as wind, birdsong, flowing water, or the sounds of people chatting and children playing. In indoor static scenarios, the movement speed is close to 0, and POIs are mainly indoor places, such as living rooms / bedrooms, offices, libraries, or cafes. The sound is stable and distinctive; for example, in a home setting, there may be sounds of a television, kitchen utensils, or pets. Learning environments may contain keyboard sounds, printer sounds, or soft conversations; coffee shop environments may contain coffee machine sounds or soft music; movement speed varies greatly depending on the type of exercise, such as jogging at 8-12 km / h, or jumping rope / exercise while moving in place; POIs are mainly sports venues, such as gyms, playgrounds, sports parks, or yoga studios, with the main sound sources being movement and rhythmic sounds, such as footsteps, breathing, the sound of gym equipment colliding, exercise music, or coaching instructions.

[0031] Therefore, based on the differences in location information and environmental audio information under different spatial scene classifications, the spatial scene classification of the user can be determined by collecting the user's location information and environmental audio information.

[0032] S103: In response to receiving a user's dialogue command, determine the dialogue topic based on spatial scene classification and the dialogue command, and conduct intelligent dialogue based on the dialogue topic.

[0033] In this embodiment, considering that users are interested in different topics under different spatial scenario classifications, for example, in commuting scenarios, users have low concentration and conversations are easily interrupted, so short, quick, and valuable topics can be selected, such as daily hot news briefs, industry news summaries, or trivia; in outdoor leisure scenarios, users' core needs are to relax and have a leisure experience, so topics of interest could be introductions to parks and attractions, nearby restaurants, or scenic route guides; in indoor static scenarios, topics of interest could be information retrieval, movie recommendations, or in-depth interactive topics; in sports and fitness scenarios, topics of interest could be exercise guidance, sports music, sports equipment, or sports supplies.

[0034] Therefore, combining spatial scene classification with user dialogue commands to determine the dialogue topic can make the dialogue topic more aligned with the user's interests.

[0035] As can be seen from the above, this embodiment takes into account that users are interested in different topics under different spatial scene categories. It determines the spatial scene category of the user based on the user's location information and environmental audio information. When receiving user instructions, it determines the dialogue topic by combining the spatial scene category and the user's dialogue instructions. This can make the dialogue topic more in line with the user's interests, thereby improving the user experience.

[0036] In one embodiment of this application, determining the spatial scene classification of the user based on location information and ambient audio information includes:

[0037] Feature extraction is performed on the location information to obtain the location feature vector;

[0038] Environmental audio information is used to extract features, resulting in audio feature vectors;

[0039] The location feature vector and the audio feature vector are weighted and fused to obtain the fused feature vector;

[0040] By inputting the fused feature vector into the classification model, the spatial scene classification of the user is obtained.

[0041] In this embodiment, key information such as longitude, latitude, POI type (e.g., subway, park), and movement speed can be extracted from location information. After normalizing the above key information, it is combined into a location feature vector. Among them, longitude and latitude can be obtained directly from GPS positioning data, POI type can be obtained by establishing a data interface with a third-party geographic information service platform, and movement speed can be calculated by first calculating the distance between two adjacent locations, calculating the time difference between the user reaching the two locations, and then dividing the above distance by the corresponding time difference.

[0042] Correspondingly, acoustic features such as Mel-frequency cepstral coefficients (MFCC), spectral entropy, short-time energy, or fundamental frequency can be extracted from environmental audio information and converted into audio feature vectors.

[0043] Based on this, the location feature vector and audio feature vector are fused into a unified fused feature vector through weighted summation. This fused feature vector is then input into the trained classification model. By learning the scene feature patterns, the model can output the spatial scene classification of the user. The classification model can be implemented using existing decision trees, support vector machines, or deep learning models.

[0044] In one embodiment of this application, both the location feature vector and the audio feature vector include features of multiple dimensions;

[0045] When weighting and fusing location feature vectors and audio feature vectors, the methods for determining the first weight corresponding to the location feature vector and the second weight corresponding to the audio feature vector include:

[0046] Acquire sample data corresponding to each of the multiple spatial scene classifications. The sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample.

[0047] Calculate the inter-class variance of each dimension feature in the location feature vector across different spatial scene classifications, and the intra-class variance of each dimension feature within the same spatial scene classification, based on multiple location feature vector samples; calculate the feature discriminative power of each dimension feature based on the inter-class and intra-class variances corresponding to each dimension feature in the location feature vector.

[0048] Calculate the inter-class variance of each dimension feature in the audio feature vector across different spatial scene classifications, and the intra-class variance of each dimension feature within the same spatial scene classification, based on multiple audio feature vector samples; calculate the feature discriminative power of each dimension feature based on the inter-class and intra-class variances corresponding to each dimension feature in the audio feature vector.

[0049] Calculate the average feature discrimination of the location feature vector based on the feature discrimination of each dimension of the location feature vector; calculate the average feature discrimination of the audio feature vector based on the feature discrimination of each dimension of the audio feature vector.

[0050] The first weight corresponding to the position feature vector and the second weight corresponding to the audio feature vector are determined based on the average feature discrimination of the position feature vector and the average feature discrimination of the audio feature vector. The first weight is positively correlated with the average feature discrimination of the position feature vector, and the second weight is positively correlated with the average feature discrimination of the audio feature vector.

[0051] In this embodiment, multiple sets of sample data for various spatial scene classifications can be pre-collected. Each set of sample data includes location feature vector samples and audio feature vector samples for the corresponding spatial scene classification. For each dimension of the location feature vector, the inter-class variance between different spatial scene classifications and the intra-class variance within the same spatial scene classification are calculated. The inter-class variance characterizes the degree of difference of the same dimension feature across different spatial scene classifications; the larger the inter-class variance for any dimension feature, the greater its contribution to distinguishing different spatial scene classifications. The intra-class variance characterizes the fluctuation of the same dimension feature within the same spatial scene classification; the smaller the intra-class variance for any dimension feature, the higher its stability within the same scene, the more concentrated its feature values, and the stronger its reliability for scene classification. Therefore, the feature discriminant power of each dimension feature in the audio feature vector can be calculated based on the inter-class and intra-class variances. The feature discriminant power comprehensively characterizes the ability of that dimension feature to distinguish multiple spatial scene classifications.

[0052] Specifically, the formula for calculating the inter-class variance of the i-th dimension feature is as follows:

[0053] ;

[0054] in,

[0055] ;

[0056] In the above calculation formula, Let represent the inter-class variance of the i-th dimension feature (feature i), and k represent the number of spatial scene categories. This represents the number of samples corresponding to the j-th spatial scene category. This represents the m-th sample corresponding to the j-th spatial scene classification. Indicates the total number of samples. Let represent the mean of feature i within the j-th spatial scene category. This represents the mean of feature i.

[0057] In the above calculation formula, As a weight, it allows scenarios with a large sample size to contribute more to inter-class differences.

[0058] Correspondingly, the formula for calculating the within-class variance of the i-th dimension feature is as follows:

[0059] ;

[0060] in,

[0061] ;

[0062] In the above calculation formula, This represents the within-class variance of the i-th dimension feature (feature i). Let the variance of the i-th dimension feature within the j-th spatial scene category be denoted as . As a weight, it can make scenarios with a large sample size contribute more to the intra-class variance.

[0063] Based on the inter-class variance and intra-class variance corresponding to the i-th dimension feature, the feature discrimination of the i-th dimension feature can be calculated using the following formula. :

[0064] ;

[0065] In the above formula, when the ratio of the inter-class variance to the intra-class variance of a certain feature i approaches infinity, that is... As it approaches infinity, the feature discrimination factor... The upper limit is (k-1), thus determining a uniform upper limit for different numbers of categories and avoiding evaluation bias caused by different numbers of scene categories.

[0066] Furthermore, the average feature discrimination of multiple dimensions in the location feature vector is calculated as the average feature discrimination of the location feature vector. Using the same method, the average feature discrimination of multiple dimensions in the audio feature vector is calculated as the average feature discrimination of the audio feature vector. .

[0067] The first weight corresponding to the position feature vector and the second weight corresponding to the audio feature vector are determined based on the average feature discrimination of the position feature vector and the average feature discrimination of the audio feature vector. For example, the first weight corresponding to the position feature vector and the second weight corresponding to the audio feature vector can be calculated using the following formula:

[0068] ;

[0069] in, Indicates the first weight. This indicates the second weight.

[0070] As can be seen from the above, this embodiment quantifies the feature discrimination of each dimension feature based on inter-class variance and intra-class variance, and determines the weights of the position feature vector and audio feature vector accordingly. This can make the fused feature vector obtained after weighted fusion more prominent in the core difference features of the spatial scene, thereby improving the classification accuracy of the spatial scene.

[0071] In one embodiment of this application, the classification model is a support vector machine model, which includes multiple binary classifiers. Each binary classifier corresponds one-to-one with multiple scene pairs, and the multiple scene pairs are obtained by combining multiple spatial scene classifications in pairs.

[0072] The classification model is trained based on sample data corresponding to multiple spatial scene classifications. The sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample.

[0073] The training process for a classification model includes:

[0074] Each location feature vector sample and its corresponding audio feature vector sample are weighted and fused to obtain a fused feature vector sample.

[0075] Perform the iterative operation multiple times until the stopping condition is met;

[0076] Each iteration operation includes:

[0077] For each fused feature vector sample, the fused feature vector sample is classified using multiple binary classifiers.

[0078] The classification results of each binary classifier are voted on according to their corresponding weights, and the classification result with the most votes is determined as the prediction result of the classification model; the weights corresponding to each binary classifier are obtained based on the similarity between the scene pairs corresponding to that binary classifier;

[0079] Calculate the classification loss based on the prediction results and the true labels of the sample data;

[0080] The model parameters of the classification model are adjusted based on the classification loss function;

[0081] The stopping condition is: the classification loss is less than a set threshold, or the number of iterations reaches a set number.

[0082] In this embodiment, all spatial scene categories can be combined in pairs to obtain multiple scene pairs, and a corresponding binary classifier can be trained based on the sample data corresponding to the two spatial scene categories in each scene pair to obtain multiple binary classifiers.

[0083] For example, all spatial scene categories include commuting / mobility scenes, outdoor leisure scenes, indoor static scenes, and sports / fitness scenes. By pairing all spatial scene categories, we can obtain six scene pairs: commuting / mobility scene-outdoor leisure scene, commuting / mobility scene-indoor static scene, commuting / mobility scene-sports / fitness scene, outdoor leisure scene-indoor static scene, outdoor leisure scene-sports / fitness scene, and indoor static scene-sports / fitness scene. Based on the sample data corresponding to the commuting / mobility scene and the outdoor leisure scene, we train a binary classifier for the commuting / mobility scene-outdoor leisure scene. Using the same method, we can train binary classifiers for each of the six scene pairs.

[0084] Based on this, multiple binary classifiers are jointly trained through multiple iterations, and the model parameters of the classification model are adjusted to obtain the trained classification model. The model parameters of the classification model may include the hyperplane parameters, kernel function parameters, and regularization parameters (C) of each binary classifier.

[0085] In each iteration, for each fused feature vector sample, multiple binary classifiers are used to classify the sample. The classification results of each classifier are then voted according to their respective weights, and the classification result with the highest number of votes is determined as the prediction result of the classification model. Considering that there may be highly similar spatial scene classifications, such as the outdoor leisure scene and the sports and fitness scene, which share multiple commonalities—including overlapping POI types (e.g., parks and sports plazas can serve as both outdoor leisure venues and venues for running, yoga, etc.) and that the environmental audio in this scene pair is based on natural sounds and sounds of crowd activity—this embodiment determines the weight of the corresponding binary classifier based on the similarity of each scene pair. Specifically, for scene pairs with a similarity greater than a similarity threshold, the weight of the corresponding binary classifier is set to a first value; for scene pairs with a similarity less than or equal to the similarity threshold, the weight of the corresponding binary classifier is set to a second value. The first value is greater than the second value. By setting differentiated weights, the model can focus more on easily confused similar scenes, improving overall classification accuracy.

[0086] For example, the classification results of the above 6 scenarios for their respective binary classifiers are shown in Table 1 below. The outdoor leisure scene - sports and fitness scene has a weight of 1.5 for the corresponding binary classifier, while the weight of the other binary classifiers is 1.

[0087] Table 1 - Classification results output by each binary classifier

[0088]

[0089] As shown in the table above, outdoor leisure scenes received the most votes. Therefore, outdoor leisure scenes will be used as the prediction result of the classification model.

[0090] As can be seen from the above, this embodiment decomposes the multi-classification problem into multiple binary classification tasks. Each binary classifier focuses on distinguishing one scene pair, which can focus on the core differences between the two scene classifications and avoid interference caused by the mixing of multiple class features. At the same time, determining the weight of the corresponding binary classifier based on the similarity of each scene pair can make the model pay more attention to easily confused similar scenes and improve the overall classification accuracy.

[0091] In one embodiment of this application, the similarity calculation method for each scene pair includes:

[0092] Acquire sample data corresponding to each of the multiple spatial scene classifications; the sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample;

[0093] For each spatial scene classification, the feature vector samples of each location and the corresponding audio feature vector samples are weighted and fused to obtain fused feature vector samples;

[0094] Calculate the average value of multiple fused feature vector samples corresponding to each spatial scene classification to obtain the average feature vector corresponding to that spatial scene classification;

[0095] Calculate the similarity between the average feature vectors corresponding to the two spatial scene classifications in each scene pair, and use this as the similarity of the scene pair.

[0096] In this embodiment, the similarity of each scene pair can be determined based on the sample data corresponding to each of the multiple spatial scene classifications, and then the weight of the corresponding binary classifier can be determined based on the similarity of each scene pair.

[0097] Specifically, for each spatial scene classification, each location feature vector sample and its corresponding audio feature vector sample can be weighted and fused to obtain a fused feature vector sample; the average value of multiple fused feature vector samples corresponding to each spatial scene classification is calculated to obtain the average feature vector corresponding to that spatial scene classification; then, using existing cosine similarity or Euclidean distance calculation methods, the similarity between the average feature vectors corresponding to the two spatial scene classifications in each scene pair is calculated as the similarity of the scene pair.

[0098] Each fused feature vector sample includes features in multiple dimensions. For each spatial scene classification, the average value of the multiple fused feature vector samples in a single dimension is calculated, and the vector composed of the average values ​​of the multiple dimensions is used as the corresponding average feature vector of the spatial scene classification.

[0099] In one embodiment of this application, determining the dialogue topic based on spatial scene classification and dialogue instructions includes:

[0100] Multiple first-candidate dialogue topics are determined based on dialogue instructions;

[0101] Multiple second candidate dialogue topics were identified based on spatial scene classification;

[0102] Calculate the intersection of multiple first-candidate dialogue topics and multiple second-candidate dialogue topics, and select a dialogue topic from the intersection.

[0103] In this embodiment, an existing bidirectional encoder-representation converter (BERT model) can be used to semantically parse the user's dialogue commands, generating multiple first candidate topics directly related to the commands. For example, if the user command is "recommend something nice to listen to", multiple first candidate topics such as "music recommendation" and "audio program recommendation" can be parsed.

[0104] Meanwhile, based on the user's spatial scene classification, a preset scene-topic mapping library can be invoked to filter out multiple second candidate topics that fit the spatial scene classification. For example, if the spatial scene classification is a commuting mobile scene, the corresponding second candidate dialogue topics could be "short-distance music," "real-time traffic conditions," or "news briefs," etc.; if the spatial scene classification is an outdoor leisure scene, the corresponding second candidate dialogue topics could be "natural sound effects," "nearby attractions," or "picnic guide," etc.

[0105] Based on this, the intersection of multiple first-candidate dialogue topics and multiple second-candidate dialogue topics is calculated, and a dialogue topic is selected from the intersection. This ensures that the selected dialogue topic matches both the user's command intent and the corresponding spatial scene classification. If the intersection is empty, a dialogue topic is selected from the multiple first-candidate dialogue topics first.

[0106] As can be seen from the above, multiple first candidate dialogue topics determined based on dialogue instructions can represent the user's instruction intent, and multiple second candidate dialogue topics determined based on spatial scene classification can filter out dialogue topics that are not applicable to the current scene. Through intersection filtering, the obtained dialogue topics can simultaneously meet the user's instruction intent and scene requirements, making the dialogue more in line with the user's current real needs.

[0107] In one embodiment of this application, selecting a dialogue topic from the intersection includes:

[0108] The first-level ranking of multiple third-candidate dialogue topics is performed based on the user's dialogue history information; the multiple third-candidate dialogue topics are the intersection of multiple first-candidate dialogue topics and multiple second-candidate dialogue topics.

[0109] If multiple third candidate dialogue topics have the same ranking, these multiple third candidate dialogue topics with the same ranking will be regarded as multiple fourth candidate dialogue topics. The multiple fourth candidate dialogue topics will be ranked in a second level based on the attention of each fourth candidate dialogue topic; the attention of the fourth candidate dialogue topics is obtained based on big data analysis.

[0110] Dialogue topics are selected from the intersection of the first-level and second-level sorting results.

[0111] In this embodiment, if there are multiple dialogue topics, i.e. multiple third candidate dialogue topics, in the intersection of multiple first candidate dialogue topics and multiple second candidate dialogue topics, the multiple third candidate dialogue topics can be sorted in the first level based on the user's dialogue history information.

[0112] Specifically, the frequency of occurrence of multiple historical dialogue topics in a user's dialogue history can be statistically analyzed. These topics are then sorted by frequency, and the top N topics are selected. For each third candidate dialogue topic, the similarity between that topic and the N historical dialogue topics is calculated, and the highest similarity is taken as the first similarity score for that third candidate dialogue topic.

[0113] Based on this, the multiple third-party candidate dialogue topics are sorted in the first level according to the corresponding first similarity from largest to smallest.

[0114] If multiple third-candidate dialogue topics have the same ranking, these topics are considered as fourth-candidate dialogue topics. Big data analysis is used to determine the level of attention these fourth-candidate dialogue topics receive; for example, higher click-through rates and longer dwell times indicate higher attention. Therefore, a second-level ranking can be performed on these fourth-candidate dialogue topics based on their level of attention.

[0115] As can be seen from the above, a user's historical dialogue information can represent the user's individual preferences, and the attention level obtained based on big data analysis can represent common preferences. In this embodiment, multiple third-party candidate dialogue topics are sorted in two levels based on historical dialogue information and attention level, which is conducive to filtering out topics that users are interested in.

[0116] In one embodiment of this application, intelligent dialogue based on a dialogue topic includes:

[0117] The dialogue content is determined based on the dialogue instructions and the dialogue topic;

[0118] The dialogue style is determined based on the user's spatial context.

[0119] Reply information is generated based on the content and style of the conversation to achieve intelligent dialogue.

[0120] In this embodiment, different dialogue styles are required for different spatial scene categories. For example, in commuting scenarios, the dialogue style can be concise and clear with high information density; in outdoor leisure scenarios, the dialogue style can be relaxed and lively. Therefore, dialogue styles corresponding to different spatial scene categories can be pre-defined. After determining the dialogue content based on dialogue instructions and topics, the dialogue style is determined based on the spatial scene category, and the response information is generated according to the dialogue style. This makes the response information more in line with the scene atmosphere, thereby improving the user experience.

[0121] Based on the same inventive concept, this application also provides an intelligent dialogue device for implementing the intelligent dialogue method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more intelligent dialogue device embodiments provided below can be found in the limitations of the intelligent dialogue method described above, and will not be repeated here.

[0122] This application provides an intelligent dialogue device, such as... Figure 2 As shown, the intelligent dialogue device 20 includes: a data acquisition module 21, a scene classification module 22, and an intelligent dialogue module 23.

[0123] Among them, the data acquisition module 21 is used to acquire the user's location information and environmental audio information;

[0124] Scene classification module 22 is used to determine the spatial scene classification of the user based on location information and environmental audio information;

[0125] The intelligent dialogue module 23 is used to respond to the user's dialogue command, determine the dialogue topic based on spatial scene classification and dialogue command, and conduct intelligent dialogue based on the dialogue topic.

[0126] In one embodiment of this application, the scene classification module 22 is specifically used for:

[0127] Based on location information and ambient audio information, the spatial scene classification of the user is determined, including:

[0128] Feature extraction is performed on the location information to obtain the location feature vector;

[0129] Environmental audio information is used to extract features, resulting in audio feature vectors;

[0130] The location feature vector and the audio feature vector are weighted and fused to obtain the fused feature vector;

[0131] By inputting the fused feature vector into the classification model, the spatial scene classification of the user is obtained.

[0132] In one embodiment of this application, both the location feature vector and the audio feature vector include features of multiple dimensions; the scene classification module 22 is further used for:

[0133] When weighting and fusing location feature vectors and audio feature vectors, the methods for determining the first weight corresponding to the location feature vector and the second weight corresponding to the audio feature vector include:

[0134] Acquire sample data corresponding to each of the multiple spatial scene classifications. The sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample.

[0135] Calculate the inter-class variance of each dimension feature in the location feature vector across different spatial scene classifications, and the intra-class variance of each dimension feature within the same spatial scene classification, based on multiple location feature vector samples; calculate the feature discriminative power of each dimension feature based on the inter-class and intra-class variances corresponding to each dimension feature in the location feature vector.

[0136] Calculate the inter-class variance of each dimension feature in the audio feature vector across different spatial scene classifications, and the intra-class variance of each dimension feature within the same spatial scene classification, based on multiple audio feature vector samples; calculate the feature discriminative power of each dimension feature based on the inter-class and intra-class variances corresponding to each dimension feature in the audio feature vector.

[0137] Calculate the average feature discrimination of the location feature vector based on the feature discrimination of each dimension of the location feature vector; calculate the average feature discrimination of the audio feature vector based on the feature discrimination of each dimension of the audio feature vector.

[0138] The first weight corresponding to the position feature vector and the second weight corresponding to the audio feature vector are determined based on the average feature discrimination of the position feature vector and the average feature discrimination of the audio feature vector. The first weight is positively correlated with the average feature discrimination of the position feature vector, and the second weight is positively correlated with the average feature discrimination of the audio feature vector.

[0139] In one embodiment of this application, the classification model is a support vector machine model, which includes multiple binary classifiers. Each binary classifier corresponds one-to-one with multiple scene pairs, and the multiple scene pairs are obtained by combining multiple spatial scene classifications in pairs.

[0140] The classification model is trained based on sample data corresponding to multiple spatial scene classifications. The sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample. During the training process of the classification model, the scene classification module 22 is further used for:

[0141] Each location feature vector sample and its corresponding audio feature vector sample are weighted and fused to obtain a fused feature vector sample.

[0142] Perform the iterative operation multiple times until the stopping condition is met;

[0143] Each iteration operation includes:

[0144] For each fused feature vector sample, the fused feature vector sample is classified using multiple binary classifiers.

[0145] The classification results of each binary classifier are voted on according to their corresponding weights, and the classification result with the most votes is determined as the prediction result of the classification model; the weights corresponding to each binary classifier are obtained based on the similarity between the scene pairs corresponding to that binary classifier;

[0146] Calculate the classification loss based on the prediction results and the true labels of the sample data;

[0147] The model parameters of the classification model are adjusted based on the classification loss function;

[0148] The stopping condition is: the classification loss is less than a set threshold, or the number of iterations reaches a set number.

[0149] In one embodiment of this application, the intelligent dialogue module 23 is specifically used for:

[0150] Multiple first-candidate dialogue topics are determined based on dialogue instructions;

[0151] Multiple second candidate dialogue topics were identified based on spatial scene classification;

[0152] Calculate the intersection of multiple first-candidate dialogue topics and multiple second-candidate dialogue topics, and select a dialogue topic from the intersection.

[0153] In one embodiment of this application, the intelligent dialogue module 23 is further configured to:

[0154] The first-level ranking of multiple third-candidate dialogue topics is performed based on the user's dialogue history information; the multiple third-candidate dialogue topics are the intersection of multiple first-candidate dialogue topics and multiple second-candidate dialogue topics.

[0155] If multiple third candidate dialogue topics have the same ranking, these multiple third candidate dialogue topics with the same ranking will be regarded as multiple fourth candidate dialogue topics. The multiple fourth candidate dialogue topics will be ranked in a second level based on the attention of each fourth candidate dialogue topic; the attention of the fourth candidate dialogue topics is obtained based on big data analysis.

[0156] Dialogue topics are selected from the intersection of the first-level and second-level sorting results.

[0157] In one embodiment of this application, the intelligent dialogue module 23 is further configured to:

[0158] The dialogue content is determined based on the dialogue instructions and the dialogue topic;

[0159] The dialogue style is determined based on the user's spatial context.

[0160] Reply information is generated based on the content and style of the conversation to achieve intelligent dialogue.

[0161] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the data acquisition module 21, scene classification module 22, and intelligent dialogue module 23 are shown.

[0162] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0163] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0164] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store preset constants such as a first weight and a second weight.

[0165] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation method described in the intelligent dialogue method provided in the embodiments of this application, or they can execute the implementation method of the electronic device described in the embodiments of this application, which will not be repeated here.

[0166] In another embodiment of this application, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0167] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., provided on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0168] Those skilled in the art will recognize that the modules / units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0169] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0170] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules, units, or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or modules / units, or it may be an electrical, mechanical, or other form of connection.

[0171] The modules / units described as separate components may or may not be physically separate. Similarly, the components shown as modules / units may or may not be physical modules / units; they may be located in one place or distributed across multiple network modules / units. Some or all of the modules / units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0172] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0173] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An intelligent dialogue method, characterized in that, include: Obtain the user's location information and ambient audio information; Based on the location information and environmental audio information, the spatial scene classification of the user is determined; In response to receiving a user's dialogue command, the system determines the dialogue topic based on the spatial scene classification and the dialogue command, and conducts intelligent dialogue based on the dialogue topic. The process of determining the spatial scene classification of the user based on the location information and environmental audio information includes: The location information is used to extract features to obtain a location feature vector; The environmental audio information is subjected to feature extraction to obtain an audio feature vector; The location feature vector and the audio feature vector are weighted and fused to obtain a fused feature vector; The fused feature vector is input into the classification model to obtain the spatial scene classification of the user. The classification model is a support vector machine model, which includes multiple binary classifiers. Each binary classifier corresponds one-to-one with multiple scene pairs. The multiple scene pairs are obtained by combining multiple spatial scene classifications in pairs. The classification model is trained based on sample data corresponding to multiple spatial scene classifications. The sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample. The training process of the classification model includes: Each location feature vector sample and its corresponding audio feature vector sample are weighted and fused to obtain a fused feature vector sample. Perform the iterative operation multiple times until the stopping condition is met; Each iteration operation includes: For each fused feature vector sample, the fused feature vector sample is classified using multiple binary classifiers. The classification results of each binary classifier are voted on according to their corresponding weights, and the classification result with the highest number of votes is determined as the prediction result of the classification model. The weight of each binary classifier is obtained based on the similarity of the scene pairs corresponding to that binary classifier. For scene pairs with similarity greater than the similarity threshold, the weight of the binary classifier corresponding to that scene pair is set to a first value. For scene pairs with similarity less than or equal to the similarity threshold, the weight of the binary classifier corresponding to that scene pair is set to a second value. Wherein, the first value is greater than the second value. Calculate the classification loss based on the prediction results and the true labels of the sample data; The model parameters of the classification model are adjusted based on the classification loss function; The stopping condition is: the classification loss is less than a set threshold, or the number of iterations reaches a set number; The similarity calculation method for each scene pair includes: Acquire sample data corresponding to each of the multiple spatial scene classifications; the sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample; For each spatial scene classification, the feature vector samples of each location and the corresponding audio feature vector samples are weighted and fused to obtain fused feature vector samples; Calculate the average value of multiple fused feature vector samples corresponding to each spatial scene classification to obtain the average feature vector corresponding to that spatial scene classification; Calculate the similarity between the average feature vectors corresponding to the two spatial scene classifications in each scene pair, and use this as the similarity of the scene pair.

2. The intelligent dialogue method as described in claim 1, characterized in that, Both the location feature vector and the audio feature vector include features in multiple dimensions; When weighting and fusing the location feature vector and the audio feature vector, the method for determining the first weight corresponding to the location feature vector and the second weight corresponding to the audio feature vector includes: Acquire sample data corresponding to each of the multiple spatial scene classifications. The sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample. Based on the multiple location feature vector samples, calculate the inter-class variance of each dimension feature in the location feature vector between different spatial scene classifications, and the intra-class variance of each dimension feature in the same spatial scene classification; calculate the feature discrimination of each dimension feature based on the inter-class variance and intra-class variance corresponding to each dimension feature in the location feature vector; Based on the multiple audio feature vector samples, calculate the inter-class variance of each dimension feature in the audio feature vector between different spatial scene classifications, and the intra-class variance of each dimension feature in the same spatial scene classification; calculate the feature discrimination of each dimension feature based on the inter-class variance and intra-class variance corresponding to each dimension feature in the audio feature vector; Based on the feature discrimination of each dimension of the position feature vector, calculate the average feature discrimination of the position feature vector; based on the feature discrimination of each dimension of the audio feature vector, calculate the average feature discrimination of the audio feature vector. The first weight corresponding to the position feature vector and the second weight corresponding to the audio feature vector are determined based on the average feature discrimination of the position feature vector and the average feature discrimination of the audio feature vector; wherein, the first weight is positively correlated with the average feature discrimination of the position feature vector and the second weight is positively correlated with the average feature discrimination of the audio feature vector.

3. The intelligent dialogue method as described in claim 1, characterized in that, The step of determining the dialogue topic based on the spatial scene classification and the dialogue instructions includes: Based on the dialogue instructions, multiple first candidate dialogue topics are determined; Based on the spatial scene classification, multiple second candidate dialogue topics are determined; Calculate the intersection of the plurality of first candidate dialogue topics and the plurality of second candidate dialogue topics, and select a dialogue topic from the intersection.

4. The intelligent dialogue method as described in claim 3, characterized in that, Selecting dialogue topics from the intersection includes: The multiple third candidate dialogue topics are sorted at the first level based on the user's dialogue history information; the multiple third candidate dialogue topics are the intersection of the multiple first candidate dialogue topics and the multiple second candidate dialogue topics. If multiple third candidate dialogue topics have the same ranking, these multiple third candidate dialogue topics with the same ranking are regarded as multiple fourth candidate dialogue topics. The multiple fourth candidate dialogue topics are then ranked in a second level based on the attention level of each of the fourth candidate dialogue topics. The attention level of the fourth candidate dialogue topics is obtained based on big data analysis. The dialogue topic is selected from the intersection of the first-level sorting results and the second-level sorting results.

5. The intelligent dialogue method as described in claim 1, characterized in that, The intelligent dialogue based on the dialogue topic includes: The dialogue content is determined based on the dialogue instructions and the dialogue topic; The dialogue style is determined based on the user's spatial context. Reply information is generated based on the dialogue content and dialogue style to achieve intelligent dialogue.

6. An intelligent dialogue device, characterized in that, include: The data acquisition module is used to acquire the user's location information and environmental audio information; The scene classification module is used to determine the spatial scene classification of the user based on the location information and environmental audio information; The intelligent dialogue module is used to respond to a user's dialogue command, determine the dialogue topic based on the spatial scene classification and the dialogue command, and conduct intelligent dialogue based on the dialogue topic; The scene classification module is specifically used for: Based on location information and ambient audio information, the spatial scene classification of the user is determined, including: Feature extraction is performed on the location information to obtain the location feature vector; Environmental audio information is used to extract features, resulting in audio feature vectors; The location feature vector and the audio feature vector are weighted and fused to obtain the fused feature vector; The fused feature vectors are input into the classification model to obtain the spatial scene classification of the user; The classification model is a support vector machine model, which includes multiple binary classifiers. Each binary classifier corresponds one-to-one with multiple scene pairs. The multiple scene pairs are obtained by combining multiple spatial scene classifications in pairs. The classification model is trained based on sample data corresponding to multiple spatial scene classifications. The sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample. During the training process of the classification model, the scene classification module is also specifically used for: Each location feature vector sample and its corresponding audio feature vector sample are weighted and fused to obtain a fused feature vector sample. Perform the iterative operation multiple times until the stopping condition is met; Each iteration operation includes: For each fused feature vector sample, the fused feature vector sample is classified using multiple binary classifiers. The classification results of each binary classifier are voted on according to their corresponding weights, and the classification result with the most votes is determined as the prediction result of the classification model. The weight of each binary classifier is obtained based on the similarity of the scene pairs corresponding to that binary classifier. For scene pairs with similarity greater than the similarity threshold, the weight of the binary classifier corresponding to that scene pair is set to the first value. For scene pairs with similarity less than or equal to the similarity threshold, the weight of the binary classifier corresponding to that scene pair is set to the second value. The first value is greater than the second value. Calculate the classification loss based on the prediction results and the true labels of the sample data; The model parameters of the classification model are adjusted based on the classification loss function; The stopping condition is: the classification loss is less than a set threshold, or the number of iterations reaches a set number; The similarity calculation method for each scene pair includes: Acquire sample data corresponding to each of the multiple spatial scene classifications; the sample data includes multiple location feature vector samples and multiple audio feature vector samples, with one location feature vector sample corresponding to one audio feature vector sample; For each spatial scene classification, the feature vector samples of each location and the corresponding audio feature vector samples are weighted and fused to obtain fused feature vector samples; Calculate the average value of multiple fused feature vector samples corresponding to each spatial scene classification to obtain the average feature vector corresponding to that spatial scene classification; Calculate the similarity between the average feature vectors corresponding to the two spatial scene classifications in each scene pair, and use this as the similarity of the scene pair.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images