User behavior data processing method and device, electronic equipment, and storage medium

Abstract

The application provides a user behavior data processing method and device, an electronic device and a storage medium, and belongs to the technical field of data processing. The method comprises the following steps: extracting audio features, semantic features and emotional features based on voice instruction data of a user; generating extended voice instruction data based on a target prompt word, the audio features, the semantic features and the emotional features; performing hierarchical clustering on the extended voice instruction data and environment data corresponding to the extended voice instruction data to obtain a plurality of first behavior data subsets; performing hierarchical clustering on touch instruction data of the user and environment data corresponding to the touch instruction data to obtain a plurality of second behavior data subsets; and extracting user behavior features based on the plurality of first behavior data subsets and the plurality of second behavior data subsets. The application can improve the processing effect of user behavior data, and further realize accurate analysis of user behavior.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing, and more particularly relates to a user behavior data processing method and device, an electronic device, and a storage medium. BACKGROUND

[0002] With the increasing popularity of intelligent devices, the interaction modes between users and various devices are increasingly diversified, generating a large amount of user behavior data. Among them, voice interaction and touch interaction are important forms of user-device interaction. However, the traditional technology still has the problem of one-sidedness when processing these user behavior data, resulting in inaccurate understanding of user intent.

[0003] Therefore, a more comprehensive and in-depth data processing method is needed to process user behavior data, so as to accurately extract user behavior features. SUMMARY

[0004] The present application aims to provide a user behavior data processing method and device, an electronic device, and a storage medium, to improve the processing effect of user behavior data, and thus realize accurate analysis of user behavior.

[0005] The first aspect of the embodiment of the present application provides a user behavior data processing method, comprising:

[0006] extracting audio features, semantic features and emotional features based on the voice instruction data of the user; generating extended voice instruction data based on the target prompt word, the audio features, the semantic features and the emotional features;

[0007] performing hierarchical clustering on the extended voice instruction data and the environment data corresponding to the extended voice instruction data to obtain a plurality of first behavior data subsets; performing hierarchical clustering on the touch instruction data of the user and the environment data corresponding to the touch instruction data to obtain a plurality of second behavior data subsets;

[0008] extracting user behavior features based on the plurality of first behavior data subsets and the plurality of second behavior data subsets.

[0009] The second aspect of the embodiment of the present application provides a user behavior data processing device, comprising:

[0010] a data augmentation module configured to extract audio features, semantic features and emotional features based on the voice instruction data of the user; and generate extended voice instruction data based on the target prompt word, the audio features, the semantic features and the emotional features;

[0011] The clustering module is configured to perform hierarchical clustering on the extended voice instruction data and environment data corresponding to the extended voice instruction data to obtain a plurality of first behavior data subsets; and perform hierarchical clustering on the touch instruction data of the user and environment data corresponding to the touch instruction data to obtain a plurality of second behavior data subsets.

[0012] The data processing module is configured to extract user behavior features based on the plurality of first behavior data subsets and the plurality of second behavior data subsets.

[0013] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the user behavior data processing method described above are implemented.

[0014] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the user behavior data processing method described above are implemented.

[0015] The user behavior data processing method and device, the electronic device, and the storage medium provided by the embodiments of the present application have the following beneficial effects. The embodiments of the present application extract audio, semantic, and emotional features from user voice instruction data, generate extended voice instruction data based on these features and target prompt words, enrich voice instruction information, more comprehensively capture user intent, avoid information loss caused by a single feature, and provide more sufficient basis for subsequent processing. The embodiments of the present application perform hierarchical clustering on the extended voice instruction data and its environment data, and the touch instruction data and its environment data respectively. This classification method considers the association between instruction data and environment data, helps to classify data with similar behavior patterns into one category, and improves the accuracy of data classification.

[0016] The embodiments of the present application extract user behavior features based on a plurality of behavior data subsets, can comprehensively analyze user behavior from different dimensions and scenarios, mine more representative and universal behavior features, provide strong support for subsequent user behavior analysis, personalized service recommendation, and other applications, and improve user experience and system performance. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 The flowchart of the user behavior data processing method provided by an embodiment of the present application is shown.

[0019] Figure 2 A structural block diagram of a user behavior data processing apparatus provided by an embodiment of the present application is shown in the figure;

[0020] Figure 3 A schematic block diagram of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0021] In the following description, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, persons skilled in the art will understand that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

[0022] In order to make the objects, technical solutions and advantages of the present application clearer, the following will be described in conjunction with the accompanying drawings and specific embodiments.

[0023] Reference should be made to Figure 1 , Figure 1 A flowchart of a user behavior data processing method provided by an embodiment of the present application is shown in the figure. The method can be executed by an electronic device. Specifically, the method can include S101-S103.

[0024] S101: Extracting audio features, semantic features and emotional features based on voice instruction data of a user; generating extended voice instruction data based on a target prompt word, audio features, semantic features and emotional features.

[0025] In the present embodiment, a user refers to a user who uses a smart device and interacts with the smart device. The smart device can include smart glasses, smart wristbands, smart phones and the like. Voice instruction data refers to voice-type instruction data generated by a user in the process of interacting with a smart device. Audio features can include physical characteristics such as pitch, volume, speed, pause interval, etc. Semantic features can include core intent, keywords, entity relationships and other content information expressed by a user. Emotional features are used to reflect the emotional state of a user in the voice, such as happiness, anger, sadness, etc.

[0026] For example, the extraction of audio features in the present embodiment can include frame processing of voice instruction data, calculation of time domain features such as short-time energy and zero-crossing rate of voice instruction data, and frequency domain features such as spectral centroid and bandwidth. The pitch parameter of the voice instruction data is extracted by a pitch detection algorithm, and the pause interval is detected by an energy threshold. The present embodiment can perform standardization processing on the extracted original features, eliminating individual pronunciation differences and environmental noise effects, for example, adjusting the volume difference by mean-variance normalization.

[0027] The extraction of the semantic feature in this embodiment can include: converting the voice instruction data into a text sequence, identifying and extracting key entities such as names, place names, and product names in the text sequence through BERT, and determining the user intent through a multi-classification model such as DistilBERT.

[0028] In this embodiment, the emotional feature is extracted based on the voice instruction data of the user, which includes: extracting acoustic emotional features and semantic emotional features based on the voice instruction data of the user, and performing weighted fusion based on the acoustic emotional features and the semantic emotional features to obtain the emotional feature.

[0029] The extraction of the emotional feature in this embodiment can include: extracting acoustic emotional features of the voice instruction data that can represent emotional states, such as pitch change rate and voice tremor degree, analyzing the energy distribution of the voice instruction data in different frequency bands, and sad emotions often have a high energy ratio in the low frequency band, while happy emotions have enhanced energy in the high frequency band. Extracting semantic emotional features of the voice instruction data that can represent emotional states. The emotional analysis results of the acoustic emotional features and the semantic emotional features are weighted and fused to obtain the final emotional feature (such as emotional category).

[0030] In this embodiment, the target prompt word is a pre-set guide word for focusing and expanding the user intent, such as "supplement", "replace", etc., and the semantic instruction data is expanded in the form of generating expanded voice instruction data. The expanded voice instruction data refers to the semantic instruction data generated by expanding on the basis of the audio features, the semantic features, and the emotional features. According to the semantic orientation of the target prompt word, the emphasis points in the audio features, the key words in the semantic features, and the tendency degrees in the emotional features can be weighted and combined to generate more complete and explicit expanded voice instruction data; or similar expanded data can be generated according to the audio features, the semantic features, and the emotional features.

[0031] S102: Hierarchical clustering is performed on the expanded voice instruction data and the environment data corresponding to the expanded voice instruction data to obtain a plurality of first behavior data subsets; and hierarchical clustering is performed on the touch instruction data of the user and the environment data corresponding to the touch instruction data to obtain a plurality of second behavior data subsets.

[0032] In the embodiment, the environmental data can include scene information when the user interacts with the smart device, such as time, location, device status, environmental noise, etc., for labeling the context of the behavior. The touch instruction data refers to the physical characteristics of the touch operation of the user, such as click position, click times, sliding track, operation duration, etc., reflecting the specific way of interaction. The first behavior data subset is the behavior grouping obtained by hierarchical clustering of the extended speech instruction data and the corresponding environmental data, for representing the scenario-based behavior mode of the user's speech interaction. The second behavior data subset is the behavior grouping obtained by hierarchical clustering of the touch instruction data and the corresponding environmental data, for representing the operation habits and scene association of the user's touch interaction.

[0033] In the embodiment, the embodiment adopts agglomerative hierarchical clustering, and each data point is regarded as an independent cluster, and then merged layer by layer according to similarity. Specifically, the embodiment can convert the speech instruction, touch operation, and environmental parameter into a multi-dimensional space vector, such as "morning commuting speech query traffic" can be represented as [time = morning, scene = commuting, intent = query, track = none]. The embodiment can perform similarity measurement on all data points, wherein the speech instruction data can focus on semantic and emotional matching, the touch instruction data can focus on operation category, and the environmental data can focus on scene parameter coincidence degree. The embodiment can merge the data points into clusters according to similarity from high to low, forming a tree structure, for example, "peak hour speech query traffic" and "peak hour speech query congestion" are clustered into one category due to similar scene and intent.

[0034] Exemplarily, taking smart glasses as an example, the acquisition of the touch instruction data by the embodiment can include: using the capacitive touch sensor built-in the glasses frame or the glasses leg to capture the user's click, sliding and other operations in real time. Among them, the click position can be located by the sensor array coordinates, and the click times are counted by the sensor trigger; the sliding track can be recorded by continuously sampling the touch point coordinates; and the operation duration can be calculated according to the time stamps of the touch start and end.

[0035] Exemplarily, the user wearing smart glasses on the morning commuting road can issue a speech instruction "query today's meeting arrangement", and record the environmental data (time, location, etc.) at that time. The user receives a call during work, slides the finger on the glasses leg touch area to answer the call (time, location, scene, etc.), and presses the recording button on the glasses leg to record. In the above process, the embodiment records the speech instruction data, touch instruction data and environmental data in real time. By dividing the speech instruction and the button instruction, the embodiment can perform hierarchical clustering on the recorded historical user behavior data to obtain the first behavior data subset and the second behavior data subset.

[0036] S103: Extract user behavior features based on the plurality of first behavior data subsets and the plurality of second behavior data subsets.

[0037] In the embodiment, the user behavior features are extracted based on the plurality of first behavior data subsets and the plurality of second behavior data subsets, including: extracting first user behavior features based on the plurality of first behavior data subsets; the first user behavior features are used to represent the voice behavior mode of the user;

[0038] extracting second user behavior features based on the plurality of second behavior data subsets; the second user behavior features are used to represent the touch behavior mode of the user;

[0039] extracting third user behavior features based on the plurality of first behavior data subsets and the plurality of second behavior data subsets; the third user behavior features are used to represent the preference features of the user for voice instructions and touch instructions.

[0040] In the embodiment, the first user behavior features are the user voice interaction rules extracted from the first behavior data subsets. The second user behavior features are the user touch operation habits extracted based on the second behavior data subsets. The third user behavior features are the selection preferences of the user for the two interaction modes in different scenarios, which are extracted by fusing the voice and touch data.

[0041] In the embodiment, the extraction of the features in the subsets can include: calculating the frequency, mean value, and proportion of the parameters in each subset, such as the frequency of the occurrence of the "road condition query" in the voice instruction in the "commuting scenario". The strong association between the behavior and the scenario is found through the conditional probability, such as that in the "night home" scenario, 80% of the touch operations are concentrated on the right side of the screen (controlling the volume / brightness). The differences between the voice and touch behaviors are compared, for example, the voice instructions are mostly used for queries and control, and the touch instructions are mostly used for adjusting the volume and recording, etc.

[0042] For example, taking a smart tablet device as an example, when the user commutes in the morning, the user often issues "query today's weather" and "play news" instructions through voice (assuming to be classified into the first behavior data subset), and the embodiment can extract the first user behavior features: the voice instructions in the morning commuting period are mostly information query type, and the tone is biased to be urgent. At the same time, the user frequently performs touch sliding fast-forward, click pause, and other operations when watching a video (assuming to be classified into the second behavior data subset), forming the second user behavior features: the touch operations in the video playing scenario are concentrated on the progress control.

[0043] The embodiment further analyzes and finds that 80% of the operation choices of the user in a quiet environment (such as a library) are voice instruction search data, and the user is more inclined to touch input to avoid interference in a noisy environment (such as a subway), which constitutes a third user behavior feature. The embodiment can flexibly switch the voice and touch interaction modes according to the noise level of the environment. Based on the above features, the tablet can automatically push weather and news voice broadcast in the morning, optimize the touch area layout in the video playing interface, and intelligently switch the optimal interaction mode when detecting the change of the environment, thereby improving the user experience.

[0044] After the above first user behavior feature, second user behavior feature and third user behavior feature are extracted, the embodiment can also optimize the user intention recognition model by using the three types of features, so that the user intention can be more accurately recognized when the user uses the instruction interaction in the future.

[0045] As can be seen from the above, the embodiment extracts audio, semantic and emotional features from the user voice instruction data, and generates extended voice instruction data based on the features and target prompt words, which can enrich the voice instruction information, more comprehensively capture the user intention, avoid information loss caused by a single feature, and provide more sufficient basis for subsequent processing. The embodiment respectively performs hierarchical clustering on the extended voice instruction data and the environment data thereof, and the touch instruction data and the environment data thereof. This classification method considers the association between the instruction data and the environment data, which is helpful to classify the data with similar behavior patterns into one category, and improves the accuracy of data classification.

[0046] The embodiment extracts user behavior features based on multiple behavior data subsets, can comprehensively analyze user behavior from different dimensions and scenes, and mine more representative and universal behavior features, thereby providing strong support for subsequent user behavior analysis, personalized service recommendation and other applications, and improving user experience and system performance.

[0047] In an embodiment of the present application, the environment data includes time data and scene data; the extended voice instruction data and the environment data corresponding to the extended voice instruction data are subjected to hierarchical clustering to obtain a plurality of first behavior data subsets, including: calculating a first data aggregation degree of the voice instruction data, calculating a second data aggregation degree of the time data, and calculating a third data aggregation degree of the scene data;

[0048] A hierarchical clustering feature sequence is determined based on the size relationship of the first data aggregation degree, the second data aggregation degree and the third data aggregation degree; the size relationship of the first data aggregation degree, the second data aggregation degree and the third data aggregation degree is positively correlated with the ordering relationship of the hierarchical clustering feature sequence;

[0049] The extended voice instruction data and the environment data are subjected to hierarchical clustering based on the hierarchical clustering feature sequence to obtain a plurality of first behavior data subsets.

[0050] In the embodiment, the hierarchical clustering is performed on the extended voice instruction data and the environment data based on a hierarchical clustering feature sequence to obtain a plurality of first behavior data subsets, including: if the hierarchical clustering feature sequence is a sequence-arranged scene feature, voice instruction feature and time feature, then: performing one-layer clustering on the extended voice instruction data and the environment data based on the scene feature to obtain a plurality of one-layer subsets;

[0051] performing two-layer clustering on the plurality of one-layer subsets based on the voice instruction feature to obtain a plurality of two-layer subsets;

[0052] performing two-layer clustering on the plurality of two-layer subsets based on the time feature to obtain a plurality of three-layer subsets;

[0053] taking the plurality of one-layer subsets, the plurality of two-layer subsets and the plurality of three-layer subsets as the plurality of first behavior data subsets;

[0054] The scene feature corresponds to the scene data, the voice instruction feature corresponds to the extended voice instruction data, and the time feature corresponds to the time data.

[0055] In the embodiment, the environment data is used to describe external conditions of a user interaction scene, and contains two types of core information: time data: specific time of interaction, which can be refined into absolute time, time period attribute, periodic feature, etc.; scene data: physical or virtual environment in which the user is located, such as geographic location, functional scene, etc. Data aggregation degree is a quantitative index for measuring similarity or correlation between data. The first data aggregation degree is used to evaluate the comprehensive similarity of the extended voice instruction data in terms of semantic features, emotional features, acoustic features, intent, etc., such as the aggregation degree of similar intent instructions. The second data aggregation degree is used to reflect the aggregation degree of the time data, such as the concentration degree of the number of instructions in a specific time period and the stability of periodic behavior. The third data aggregation degree is used to reflect the aggregation degree of the scene data, such as the consistency of behavior in the same geographic location and environment state.

[0056] The clustering priority formed according to the data aggregation degree in the embodiment determines the data hierarchical processing order, such as "scene-voice-time", which means that the scene clustering is preferred, then the voice instruction is subdivided, and finally the time factor is considered.

[0057] The hierarchical clustering feature sequence is a clustering priority order formed according to the data aggregation degree from large to small, which determines the execution order of different data types in hierarchical clustering. The first data aggregation degree, the second data aggregation degree and the third data aggregation degree are sorted from large to small in value, and the order is corresponding to the order of the scene feature, the voice instruction feature and the time feature in the hierarchical clustering feature sequence. For example, if the aggregation degree of the scene data is the highest, the scene feature corresponding to the scene data is in the first order in the hierarchical clustering feature sequence, and the scene feature is preferred as the first layer clustering basis for clustering data.

[0058] In this embodiment, the first aggregation degree can be calculated by semantic similarity (or intention similarity), sentiment consistency, and acoustic feature similarity. The distribution rule of the time data can be analyzed, such as calculating the standard deviation of the number of instructions in a specific period, and the aggregation degree is calculated by the standard deviation of the number of instructions in different time periods. The smaller the standard deviation is, the higher the aggregation degree is. The third aggregation degree can be calculated based on the co-occurrence frequency of scene labels or the similarity of environmental parameters.

[0059] The data type with a higher aggregation degree has a higher priority in the hierarchical clustering feature sequence. For example, if the scene data aggregation degree is significantly higher than the time data, the clustering order is scene, voice, and time. The embodiment can cluster data with high similarity first, which can reduce the subsequent calculation complexity and improve the clustering accuracy. For example, the data is divided by scene first, the voice instruction commonality in the scene is analyzed, and finally the behavior difference in different time periods is refined.

[0060] The hierarchical clustering of the embodiment can include: one layer of clustering (scene dimension): aggregating voice instructions and environmental data in the same or similar scene, for example, all voice instructions in the "commuting" scene are classified into a subset. Two layers of clustering (voice dimension): in the scene subset, the voice instructions are further subdivided according to the intention, emotion, and other features of the voice instructions, such as classifying the voice instructions in the "commuting" subset into smaller subsets such as "query traffic" and "play music". Three layers of clustering (time dimension): for each voice instruction subset, the time rule is cut, for example, the "query traffic" instruction is further divided into "early peak query" and "late peak query" subsets.

[0061] For example, the user generates the following voice instruction data and environmental data during driving:

[0062] Instruction A: On Monday morning at 7:45, say "check the road ahead" in a congested section, and the environment is noisy and moving.

[0063] Instruction B: On Friday evening at 18:30, say "play Z's song" in the parking lot, and the environment is quiet and stationary.

[0064] Instruction C: On Tuesday morning at 7:50, say "navigate to xx" on the expressway, and the environment is noisy and moving.

[0065] The embodiment analyzes and finds that the voice instructions (instructions A and C) in the "driving" scene have high similarity in semantics (query, navigation) and environmental state, and the third data aggregation degree (scene) is the highest. The instruction frequency in the morning 7-8 o'clock is significantly higher than that in other periods, and the second data aggregation degree (time) is the second. The intention of the voice instruction is quite different, and the first data aggregation degree (voice) is the lowest. Therefore, the clustering order is determined as "scene-time-voice".

[0066] One layer clustering (scenario): instructions A, B, C are divided into two subsets of "driving" and "parking", and instructions A, C are classified into the "driving" scenario.

[0067] Two layer clustering (time): in the "driving" subset, it is divided into "morning commute" (instructions A, C) and "evening non-commute" (no corresponding instructions) according to time.

[0068] Three layer clustering (voice): in the "morning commute" subset, instruction A is classified as "road condition query" and instruction C is classified as "navigation" according to intent.

[0069] From the above, the embodiment determines the clustering priority by quantifying the data aggregation degree, and combines the environmental data to perform hierarchical clustering, thereby accurately analyzing the user behavior mode. The embodiment not only improves the data processing efficiency and reduces the computational complexity, but also mines the user demand law based on the scene, voice, and time, helps the intelligent device to provide personalized and proactive services that are more in line with user habits, and significantly optimizes the user interaction experience.

[0070] In an embodiment of the present application, the first data aggregation degree of the voice instruction data is calculated, comprising:

[0071] Based on the semantic features extracted from the voice instruction data, the user behavior intent and the user intent category to which the user behavior intent belongs are determined;

[0072] The intent aggregation degree of the user behavior intent and the intent category aggregation degree of the user intent category are calculated;

[0073] The intent aggregation degree and the intent category aggregation degree are weighted and summed to obtain the first data aggregation degree.

[0074] In the embodiment, the intent aggregation degree of the user behavior intent is calculated, specifically comprising: clustering the user behavior intent extracted from the voice instruction data based on the K-means algorithm to obtain a clustering result, and determining the intent aggregation degree according to the clustering result.

[0075] In the embodiment, the intent category aggregation degree of the user intent category is calculated, specifically comprising: calculating the frequency of each user intent category in the voice instruction data, and calculating the intent category aggregation degree based on the number of user intent categories and the frequency of each user intent category.

[0076] In the embodiment, the user behavior intention refers to the specific demand expressed by the user through the voice instruction, such as “query tomorrow's weather” and the like. The user intention category refers to the abstract classification of the user behavior intention, such as the query category, the control category, the entertainment category and the like. The intention aggregation degree is used to measure the similarity degree between the deep intention, and reflects the commonality intensity of the specific intention. The intention category aggregation degree is used to measure the concentration degree of the shallow intention category in the data set, such as the appearance frequency of the query category intention.

[0077] In the embodiment, the embodiment can analyze the semantic features of the voice instruction through the natural language processing technology, determine the user behavior intention and the category thereof. For example, “play Z's song” is classified as the “entertainment category-music playing” intention.

[0078] The embodiment can use the K-means algorithm to aggregate the user behavior intentions with the similarity reaching the preset threshold into subcategories, such as aggregating “play Z's song” and “play L's song” into the “music playing” subcategory. The less the number of subcategories is, and the higher the similarity of the intentions in the category is, the higher the intention aggregation degree is.

[0079] The embodiment can assign weights to the intention aggregation degree and the intention category aggregation degree according to the demand. For example, if the high-frequency common behavior is focused on, the weight of the intention category aggregation degree can be increased; if the detail difference of the specific intention is focused on, the intention aggregation degree is focused on.

[0080] For example, the user issues multiple voice instructions to the smart glasses: “when will my order arrive” “query logistics information” “play Z's song” “help me cancel this order”. The embodiment can first identify the intention and the category: the first two are “query category-order query”, the third is “entertainment category-music playing”, and the fourth is “control category-order operation”. Among them, the smart glasses are connected with the user's other smart devices (such as mobile phones) through Bluetooth, and the user can control the operation of the mobile phone through the smart glasses, such as playing music, answering the phone, querying information and the like.

[0081] Through the K-means algorithm, the two order query intentions are aggregated into one category, the music playing and order operation intentions are respectively classified, and the intention aggregation degree is calculated according to the number of subcategories and the number of similar intentions in the subcategory. The frequency of each of the query category, the control category and the entertainment category is calculated, and the intention category aggregation degree is determined according to the frequency distribution. After the intention aggregation degree and the intention category aggregation degree are weighted and summed, the first data aggregation degree is obtained.

[0082] The embodiment mines the deep commonality of the intention through the K-means clustering, analyzes the shallow distribution in combination with the intention category frequency, and quantifies the voice instruction aggregation degree after weighted fusion. The embodiment can not only capture the high-frequency intention mode, but also identify the specific intention details, thereby providing multi-dimensional data support for the clustering analysis, and improving the accuracy and comprehensiveness of the user behavior pattern analysis.

[0083] In an embodiment of the present application, the extended voice instruction data is generated based on the target prompt word, the audio feature, the semantic feature and the emotion feature, comprising: generating a plurality of extended text queries based on the target prompt word and the semantic feature;

[0084] The extended voice instruction data is generated based on the plurality of extended text queries, the audio feature and the emotion feature.

[0085] In the embodiment, the target prompt word includes a similar replacement prompt word and an information completion prompt word;

[0086] The plurality of extended text queries are generated based on the target prompt word and the semantic feature, comprising: generating a first extended text query based on the information completion prompt word and the semantic feature;

[0087] The first similar text query is generated based on the similar replacement prompt word and the semantic feature;

[0088] The second similar text query is generated based on the similar replacement prompt word and the first extended text query;

[0089] The first extended text query, the first similar text query and the second similar text query are taken as the plurality of extended text queries.

[0090] In the embodiment, the plurality of extended text queries are generated based on the target prompt word and the semantic feature, further comprising: generating a second extended text query based on the information completion prompt word and the first similar text query, and taking the second extended text query as the plurality of extended text queries.

[0091] In the embodiment, the target prompt word is a keyword guiding the expansion direction of the voice instruction, for example, the target prompt word is a guiding parameter for model input, which does not directly generate content, but indicates the model to expand the original instruction based on the semantic feature.

[0092] The information completion prompt word is a sceneized label for the model to supplement details, which is used to guide the model to supplement the missing information of the user instruction. For example, the information completion prompt word can be {"time completion":"semantic feature", "place completion":"semantic feature", "attribute completion":"semantic feature"}, which guides the model to extract the missing elements from the semantic feature and fill them in.

[0093] The similar replacement prompt word is a semantic guiding label for the model to perform synonymous conversion, which is used to guide the model to generate expression replacement with similar semantics. For example, the similar replacement prompt word can be {"verb replacement":"semantic feature", "noun replacement":"semantic feature", "adjective replacement":"semantic feature"}, which guides the model to generate different expressions with equivalent semantics.

[0094] In this embodiment, the extended text query is a derivative text generated based on the original voice instruction, which contains four types: the first extended text query: the text obtained by adding details to the original instruction through information completion prompt words, such as supplementing time, place and other conditions; the first similar text query: the text obtained by using similar replacement prompt words to replace the original instruction; the second similar text query: the text obtained by replacing the extended instruction with similar / synonymous replacement; and the second extended text query: the text obtained by extending the similar replacement text.

[0095] The extended voice instruction data refers to the fusion of text, audio, and emotion multi-dimensional features, which converts the extended text query into more rich voice instruction expression.

[0096] For example, the original instruction is "check the weather", and the corresponding first extended text query can be "check today's weather", "query today's weather", "query the current weather", etc.; the first similar text query can be "search weather"; the second similar text query can be "search today's weather", "search the current weather", etc.; the third similar text query can be "understand weather conditions"; and the second extended text query can be "understand today's weather conditions", etc.

[0097] In this embodiment, the original voice instruction is converted into a semantic vector (such as a word embedding generated by BERT), and keywords, entities, and intents are extracted.

[0098] This embodiment uses similar replacement prompt words to guide the model (such as GPT-3) to find synonymous expressions in the semantic space. This embodiment uses information completion prompt words to trigger the model to extract missing information from external knowledge bases or context. The model generates the first extended text (completion) and the first similar text (replacement) based on the semantic features of the original instruction and the target prompt words. The model applies the target prompt words to the basic extension result again to generate the second similar text (replacement after extension) and the second extended text (replacement after completion), forming a semantic variant cluster.

[0099] For example, the user asks the intelligent customer service voice: "How come my order hasn't arrived yet", and this embodiment can generate an extended instruction through the following process:

[0100] Semantic feature extraction: keywords "order" and "not arrived", intent category "query type-logistics query", and missing entity "expected delivery time".

[0101] Input the similar replacement prompt words, information completion prompt words, and semantic features into the model to obtain:

[0102] The first extended text: "My latest logistics order hasn't arrived yet";

[0103] The first similar text is: "Inquire about the delivery time and the reason for non-delivery of my order";

[0104] The second similar text is: "Inquire about the delivery time and the reason for non-delivery of my latest logistics order";

[0105] The second expanded text is: "My latest logistics order has not been received yet, please inquire about the specific delivery time and the reason for non-delivery";

[0106] The first expanded text query, the first similar text query, the second similar text query, and the second expanded text query are multiple expanded text queries. According to the speed, tone, timbre, emotion, and other characteristics of the original voice instruction data, the embodiment can generate expanded voice instruction data for all expanded text queries.

[0107] The embodiment can cover the user's multi-dimensional expression needs and improve the fault tolerance rate of voice instruction recognition, and can ensure that the expanded instruction is consistent with the original interaction style, thereby significantly enhancing the accuracy and user experience of the intelligent interaction system.

[0108] The user behavior data processing method corresponding to the above embodiment, Figure 2 The structure block diagram of the user behavior data processing device provided by an embodiment of the present application is shown. For ease of illustration, only parts related to the embodiments of the present application are shown. Reference Figure 2 The user behavior data processing device 20 includes a data expansion module 21, a clustering module 22, and a data processing module 23.

[0109] The data expansion module 21 is configured to extract audio features, semantic features, and emotional features based on the voice instruction data of the user; and generate expanded voice instruction data based on the target prompt word, the audio features, the semantic features, and the emotional features;

[0110] The clustering module 22 is configured to perform hierarchical clustering on the expanded voice instruction data and the environment data corresponding to the expanded voice instruction data to obtain a plurality of first behavior data subsets; and perform hierarchical clustering on the touch instruction data of the user and the environment data corresponding to the touch instruction data to obtain a plurality of second behavior data subsets;

[0111] The data processing module 23 is configured to extract user behavior features based on the plurality of first behavior data subsets and the plurality of second behavior data subsets.

[0112] In an embodiment of the present application, the environment data includes time data and scene data; the clustering module 22 is specifically configured to calculate a first data aggregation degree of the voice instruction data, calculate a second data aggregation degree of the time data, and calculate a third data aggregation degree of the scene data;

[0113] determine a hierarchical clustering feature sequence based on a size relationship of the first data aggregation degree, the second data aggregation degree and the third data aggregation degree; the size relationship of the first data aggregation degree, the second data aggregation degree and the third data aggregation degree is positively correlated with an order relationship of the hierarchical clustering feature sequence;

[0114] perform hierarchical clustering on the extended voice instruction data and the environmental data based on the hierarchical clustering feature sequence, to obtain a plurality of first behavior data subsets.

[0115] In an embodiment of the present application, the clustering module 22 is specifically further configured to, if the hierarchical clustering feature sequence is a sequence of the scene feature, the voice instruction feature and the time feature: perform one-layer clustering on the extended voice instruction data and the environmental data based on the scene feature, to obtain a plurality of one-layer subsets;

[0116] perform two-layer clustering on the plurality of one-layer subsets based on the voice instruction feature, to obtain a plurality of two-layer subsets;

[0117] perform two-layer clustering on the plurality of two-layer subsets based on the time feature, to obtain a plurality of three-layer subsets;

[0118] take the plurality of one-layer subsets, the plurality of two-layer subsets and the plurality of three-layer subsets as the plurality of first behavior data subsets;

[0119] the scene feature corresponds to the scene data, the voice instruction feature corresponds to the extended voice instruction data, and the time feature corresponds to the time data.

[0120] In an embodiment of the present application, the clustering module 22 is specifically further configured to determine a user behavior intention and a user intention category to which the user behavior intention belongs based on a semantic feature extracted from the voice instruction data; calculate an intention aggregation degree of the user behavior intention and an intention category aggregation degree of the user intention category; and perform weighted summation on the intention aggregation degree and the intention category aggregation degree, to obtain the first data aggregation degree.

[0121] In an embodiment of the present application, the data expansion module 21 is specifically configured to generate a plurality of extended text queries based on the target prompt word and the semantic feature;

[0122] generate the extended voice instruction data based on the plurality of extended text queries, the audio feature and the sentiment feature.

[0123] In an embodiment of the present application, the target prompt word includes a similar replacement prompt word and an information completion prompt word; and the data expansion module 21 is specifically further configured to generate a first expanded text query based on the information completion prompt word and the semantic feature.

[0124] generate a first similar text query based on the similar replacement prompt word and the semantic feature;

[0125] A second similar text query is generated based on similar replacement suggestions and the first expanded text query;

[0126] The first extended text query, the first similar text query, and the second similar text query are treated as multiple extended text queries.

[0127] In one embodiment of this application, the data processing module 23 is specifically used to extract first user behavior features based on multiple first behavior data subsets; the first user behavior features are used to characterize the user's voice behavior pattern.

[0128] Second user behavior features are extracted based on multiple subsets of second behavior data; these second user behavior features are used to characterize the user's touch behavior patterns.

[0129] A third user behavior feature is extracted based on multiple subsets of first behavior data and multiple subsets of second behavior data; the third user behavior feature is used to characterize the user's preference for voice commands and touch commands.

[0130] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example... Figure 2 The functions of the data expansion module 21, clustering module 22, and data processing module 23 are shown.

[0131] It should be appreciated that in the embodiments of the present application, the processor 301 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0132] The input device 302 can include a touchpad, a fingerprint collection sensor (for collecting fingerprint information and direction information of a fingerprint of a user), a microphone, etc., and the output device 303 can include a display (LCD, etc.), a speaker, etc.

[0133] The memory 304 can include a read-only memory and a random access memory, and provide instructions and data for the processor 301. A portion of the memory 304 can also include a non-volatile random access memory. For example, the memory 304 can also store information of user data.

[0134] In specific implementations, the processor 301, the input device 302 and the output device 303 described in the embodiments of the present application can perform the implementation manners described in the embodiments of the user behavior data processing method provided by the embodiments of the present application, and can also perform the implementation manners of the electronic device 300 described in the embodiments of the present application, which will not be described here.

[0135] In another embodiment of the present application, a computer readable storage medium is provided, which stores a computer program. The computer program includes program instructions, which, when executed by a processor, implement all or part of the processes of the above-mentioned embodiment methods. The computer program can also instruct related hardware to complete the implementation. The computer program can be stored in a computer readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0136] The computer readable storage medium can be an internal storage unit of the electronic device, such as a hard disk or a memory of the electronic device. The computer readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit and the external storage device of the electronic device. The computer readable storage medium is used to store the computer program and other programs and data required by the electronic device. The computer readable storage medium can also be used to temporarily store data that has been output or will be output.

[0137] Those skilled in the art can appreciate that the modules / units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic device and the units described above can refer to the corresponding processes in the above-mentioned method embodiments, which will not be described here.

[0139] In several embodiments provided in the present application, it should be understood that the disclosed electronic device and method can be implemented in other manners. For example, the division of the above-described apparatus embodiments is merely an example, and there can be other division manners. For example, multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, or can be in electrical, mechanical or other forms.

[0140] The modules or units illustrated as separate parts can or can not be physically separate, and the parts illustrated as modules or units can or can not be physical modules or units, i.e., can be located in one place, or can be distributed on multiple network modules or units. Some or all of the modules or units can be selected according to actual needs to achieve the purposes of the embodiments of the present application.

[0141] In addition, each functional module / unit in each embodiment of the present application can be integrated into a processing module / unit, or each module / unit can exist physically separately, or two or more modules / units can be integrated into one module / unit. The integrated module / unit can be implemented in the form of hardware or in the form of a software functional module / unit.

[0142] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for processing user behavior data, characterized in that, include: Extracting audio features, semantic features, and emotional features from user voice command data; Extended voice command data is generated based on the target prompt words, the audio features, the semantic features, and the emotional features; Hierarchical clustering is performed on the extended voice command data and the environmental data corresponding to the extended voice command data to obtain multiple first row data subsets; Hierarchical clustering is performed on the user's touch command data and the environmental data corresponding to the touch command data to obtain multiple second row data subsets; User behavior features are extracted based on the plurality of first behavior data subsets and the plurality of second behavior data subsets; The environmental data includes time data and scene data; The hierarchical clustering of the extended voice command data and the corresponding environmental data yields multiple first-row data subsets, including: Calculate the first data clustering degree of the voice command data, calculate the second data clustering degree of the time data, and calculate the third data clustering degree of the scene data; The hierarchical clustering feature sequence is determined based on the relationship between the first data clustering degree, the second data clustering degree, and the third data clustering degree; the relationship between the first data clustering degree, the second data clustering degree, and the third data clustering degree is positively correlated with the ranking relationship of the hierarchical clustering feature sequence; Based on the hierarchical clustering feature sequence, hierarchical clustering is performed on the extended voice command data and the environmental data to obtain multiple first row data subsets; The hierarchical clustering of the extended voice command data and the environmental data based on the hierarchical clustering feature sequence yields multiple first-row data subsets, including: If the hierarchical clustering feature sequence consists of sequentially arranged scene features, voice command features, and time features, then: Based on the scene features, the extended voice command data and the environmental data are clustered in one layer to obtain multiple one-layer subsets; Based on the voice command features, perform two-level clustering on the multiple first-level subsets to obtain multiple second-level subsets; Based on the time features, perform two-level clustering on the multiple two-level subsets to obtain multiple three-level subsets; The plurality of first-level subsets, the plurality of second-level subsets, and the plurality of third-level subsets are used as the plurality of first-row data subsets; The scene feature corresponds to scene data, the voice command feature corresponds to extended voice command data, and the time feature corresponds to time data.

2. The user behavior data processing method as described in claim 1, characterized in that, The calculation of the first data clustering degree of the voice command data includes: The user's behavioral intent and the user intent category to which the user's behavioral intent belongs are determined based on the semantic features extracted from the voice command data. Calculate the intent clustering degree of the user's behavioral intent, and calculate the intent category clustering degree of the user's intent category; The first data clustering degree is obtained by weighted summation of the intent clustering degree and the intent category clustering degree.

3. The user behavior data processing method as described in claim 1, characterized in that, The process of generating extended voice command data based on target prompts, audio features, semantic features, and emotional features includes: Multiple extended text queries are generated based on the target prompt words and the semantic features; Extended voice command data is generated based on the multiple extended text queries, the audio features, and the emotional features.

4. The user behavior data processing method as described in claim 3, characterized in that, The target prompts include similar replacement prompts and information completion prompts; The generation of multiple extended text queries based on the target prompt words and the semantic features includes: A first expanded text query is generated based on the information completion prompts and the semantic features; A first similar text query is generated based on the similar replacement prompts and the semantic features; A second similar text query is generated based on the similar replacement suggestion words and the first expanded text query; The first extended text query, the first similar text query, and the second similar text query are used as the plurality of extended text queries.

5. The user behavior data processing method as described in claim 1, characterized in that, The extraction of user behavior features based on the plurality of first behavior data subsets and the plurality of second behavior data subsets includes: A first user behavior feature is extracted based on the plurality of first behavior data subsets; the first user behavior feature is used to characterize the user's voice behavior pattern. Second user behavior features are extracted based on the multiple subsets of second behavior data; the second user behavior features are used to characterize the user's touch behavior patterns. A third user behavior feature is extracted based on the plurality of first behavior data subsets and the plurality of second behavior data subsets; the third user behavior feature is used to characterize the user's preference features for voice commands and touch commands.

6. A user behavior data processing device, characterized in that, include: The data augmentation module is used to extract audio features, semantic features, and emotional features based on the user's voice command data; Extended voice command data is generated based on the target prompt words, the audio features, the semantic features, and the emotional features; The clustering module is used to perform hierarchical clustering on the extended voice command data and the corresponding environmental data to obtain multiple first-row data subsets; and to perform hierarchical clustering on the user's touch command data and the corresponding environmental data to obtain multiple second-row data subsets; the environmental data includes time data and scene data; The clustering module is specifically used to calculate the first data clustering degree of the voice command data, the second data clustering degree of the time data, and the third data clustering degree of the scene data; The hierarchical clustering feature sequence is determined based on the relationship between the first data clustering degree, the second data clustering degree, and the third data clustering degree. The magnitudes of the first data clustering degree, the second data clustering degree, and the third data clustering degree are positively correlated with the ordering relationship of the hierarchical clustering feature sequence; Based on the hierarchical clustering feature sequence, hierarchical clustering is performed on the extended voice command data and the environmental data to obtain multiple first row data subsets; The clustering module is further configured to, if the hierarchical clustering feature sequence consists of sequentially arranged scene features, voice command features, and time features, then: Based on the scene features, the extended voice command data and the environmental data are clustered in one layer to obtain multiple one-layer subsets; Based on the voice command features, perform two-level clustering on the multiple first-level subsets to obtain multiple second-level subsets; Based on the time features, perform two-level clustering on the multiple two-level subsets to obtain multiple three-level subsets; The plurality of first-level subsets, the plurality of second-level subsets, and the plurality of third-level subsets are used as the plurality of first-row data subsets; The scene feature corresponds to scene data, the voice command feature corresponds to extended voice command data, and the time feature corresponds to time data; The data processing module is used to extract user behavior features based on the plurality of first behavior data subsets and the plurality of second behavior data subsets.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images