Conversation-based intention recognition method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a dialogue-based intention recognition method and device, electronic equipment and a storage medium, belongs to the technical field of artificial intelligence, and is applied to the field of financial science and technology and the field of health medical treatment. The method comprises the following steps: acquiring conversation voice between a target object and a service agent, wherein the conversation voice comprises at least one voice segment; performing voice feature extraction on the voice segments to obtain voice rhythm features and time sequence behavior features; performing voice recognition on the voice segment to obtain a dialogue text, and performing text feature extraction on the dialogue text to obtain a text semantic feature; performing attention feature extraction according to the voice rhythm features, the time sequence behavior features, the text semantic features and preset historical dialogue features to obtain context attention features; and performing intention recognition on the target object according to the voice rhythm feature, the time sequence behavior feature, the text semantic feature and the context attention feature. The intention recognition method and device can improve intention recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and is applied to the fields of financial technology and health care, and particularly relates to a dialogue-based intent recognition method and device, an electronic device and a storage medium. BACKGROUND

[0002] In a service interaction process, by performing intent recognition on a user, the service strategy of a service agent can be optimized, thereby improving the conversion rate of business services. Taking an insurance business in a financial technology scenario as an example, if the insurance service system identifies that the insurance intent of a customer is low, the service agent can be immediately pushed an incentive tool such as a premium coupon or a time-limited free medical examination, and the incentive tool is real-time pushed to the customer by the service agent, so as to improve the insurance conversion rate. Taking an online reconsultation business in a health care scenario as an example, if the medical service system identifies that the reconsultation intent of a patient is low, the service agent can be immediately pushed a patient service strategy, so that the service agent pushes a medical insurance reimbursement service or a weekend free queuing reservation service to the patient according to the patient service strategy, so as to improve the reconsultation conversion rate.

[0003] In related technologies, intent recognition is usually performed by using keyword matching, sentiment polarity judgment and the like. However, in a service interaction, a user often does not directly express the real intent, and these methods can only capture the literal meaning of a dialogue, and cannot deeply understand the dialogue between the user and the service agent, resulting in low accuracy of intent recognition. SUMMARY

[0004] To overcome the problem of low accuracy of intent recognition in related technologies, the main purpose of the embodiments of the present application is to propose a dialogue-based intent recognition method and device, an electronic device and a storage medium, aiming to improve the accuracy of intent recognition.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application proposes a dialogue-based intent recognition method, which comprises: obtaining dialogue voice between a target object and a service agent; wherein the dialogue voice comprises at least one voice segment; performing voice feature extraction on the voice segment to obtain voice prosody features and timing behavior features; performing voice recognition on the voice segment to obtain dialogue text, and performing text feature extraction on the dialogue text to obtain text semantic features; performing attention feature extraction according to the voice prosody features, the timing behavior features, the text semantic features and preset historical dialogue features to obtain context attention features; wherein the historical dialogue features are obtained by performing feature extraction on historical dialogue voice between the target object and the service agent, and the dialogue end time of the historical dialogue voice is before the dialogue start time of the dialogue voice. Intent recognition is performed on the target object based on the speech prosody features, the temporal behavior features, the text semantic features, and the contextual attention features.

[0006] In some embodiments, the historical dialogue speech includes historical speech segments, and the historical dialogue features include segment dialogue features of the historical speech segments. The step of extracting attention features based on the speech prosody features, the temporal behavior features, the text semantic features, and preset historical dialogue features to obtain contextual attention features includes: A query vector is calculated based on the prosodic features, temporal behavior features, and text semantic features of each speech segment; Calculate the key vector and value vector based on the features of the described dialogue segments; Attention is calculated based on the query vector, the key vector, and the value vector to obtain segment attention features; The fragment attention features are fused to obtain the context attention features.

[0007] In some embodiments, the intention recognition of the target object based on the speech prosody features, the temporal behavior features, the text semantic features, and the contextual attention features includes: For each speech segment, the speech prosody features, temporal behavior features, and text semantic features are weighted and fused to obtain segment fusion features; Feature fusion is performed on each of the fragment fusion features to obtain the first fusion feature; The first fusion feature and the context attention feature are weighted and fused to obtain the second fusion feature; Intent recognition is performed on the target object based on the second fusion feature.

[0008] In some embodiments, the historical dialogue features include historical prosodic features, historical temporal features, and historical text features. The weighted fusion of the prosodic features, temporal behavioral features, and textual semantic features to obtain fragment fusion features includes: Attention is calculated based on the historical prosodic features and the speech prosodic features to obtain prosodic weights, and the prosodic weights are multiplied by the speech prosodic features to obtain weighted prosodic features. Attention is calculated based on the historical time series features and the time series behavior features to obtain time series weights. The time series weights are then multiplied by the time series behavior features to obtain weighted time series features. Attention is calculated based on the historical text features and the text semantic features to obtain semantic weights, and the semantic weights are multiplied by the text semantic features to obtain weighted text features; The weighted prosodic features, the weighted temporal features, and the weighted text features are summed to obtain the fragment fusion features.

[0009] In some embodiments, the weighted fusion of the first fusion feature and the contextual attention feature to obtain the second fusion feature includes: Based on the speech prosody features, the temporal behavior features, the text semantic features, and the context attention features, the fusion weight of the first fusion feature is calculated, and the fusion weight is multiplied by the first fusion feature to obtain the weighted fusion feature; Obtain the dialogue stage of the spoken dialogue; The attention weights of the contextual attention features are calculated based on the dialogue phase, and the attention weights are multiplied by the contextual attention features to obtain weighted attention features; The weighted fusion feature and the weighted attention feature are added together to obtain the second fusion feature.

[0010] In some embodiments, the step of performing intent recognition on the target object based on the second fusion feature includes: Spatiotemporal feature extraction is performed on the second fusion feature to obtain spatiotemporal attention features; Intent recognition is performed on the target object based on the spatiotemporal attention features.

[0011] In some embodiments, after performing intent recognition on the target object based on the speech prosody features, the temporal behavior features, the text semantic features, and the contextual attention features, the method further includes: Obtain the intent category obtained by performing intent recognition on the target object; Sentiment analysis is performed based on the prosodic features and semantic features of the text to obtain the sentiment category of the target object; The target scripts for the service agents are selected from a preset script library based on the emotion category and the intent category. In response to receiving the target object's feedback information on the target script, the service agent generates a follow-up script based on the feedback information.

[0012] To achieve the above objectives, a second aspect of this application provides a dialogue-based intent recognition device, the device comprising: The voice acquisition module is used to acquire the voice dialogue between the target object and the service agent; wherein the voice dialogue includes at least one voice segment. The speech feature extraction module is used to extract speech features from the speech segment to obtain speech prosody features and temporal behavior features; The text feature extraction module is used to perform speech recognition on the speech segment to obtain the dialogue text, and to extract text features from the dialogue text to obtain text semantic features. The attention feature extraction module is used to extract attention features based on the speech prosody features, the temporal behavior features, the text semantic features, and the preset historical dialogue features to obtain contextual attention features; wherein, the historical dialogue features are obtained by feature extraction from the historical dialogue voice between the target object and the service agent, and the dialogue end time of the historical dialogue voice is before the dialogue start time of the dialogue voice. The intent recognition module is used to recognize the intent of the target object based on the speech prosody features, the temporal behavior features, the text semantic features, and the contextual attention features.

[0013] To achieve the above objectives, a third aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0014] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in the first aspect.

[0015] The dialogue-based intent recognition method, device, electronic device, and computer-readable storage medium proposed in this application acquire the dialogue voice between a target and a service agent to recognize the target's intent based on the dialogue voice. The dialogue voice includes at least one voice segment. To better understand the deep semantics of the dialogue voice, voice features are extracted from the voice segments to obtain prosodic features and temporal behavioral features. Prosodic features reflect the target's voice rhythm, intonation, and other characteristics, thereby determining the target's emotion, attitude, and intent intensity. Temporal behavioral features reflect the dynamic changes in the dialogue voice over time, capturing the procedural information of the target's intent expression. Dialogue logic relies on grammar, sentence structure, and conjunctions. Voice feature extraction can only capture non-textual cues in the dialogue voice and cannot accurately capture the dialogue semantics. To understand complex intents, voice recognition is performed on the voice segments to obtain the dialogue text, and textual features are extracted from the dialogue text to obtain textual semantic features, which are then used to understand the content of the dialogue voice. The intent of a target is often implicit in dynamically changing contextual information. To capture key contextual information, attention features are extracted based on phonological prosody features, temporal behavioral features, textual semantic features, and historical dialogue features to obtain contextual attention features. These contextual attention features upgrade intent recognition from literal matching to contextual understanding. By combining phonological prosody features, temporal behavioral features, textual semantic features, and contextual attention features to identify the target's intent, and by combining multimodal features, the complex intents implicit in dialogue speech can be understood, improving the accuracy of intent recognition. Attached Figure Description

[0016] Figure 1 This is a flowchart of the dialogue-based intent recognition method provided in the embodiments of this application; Figure 2 yes Figure 1 The flowchart of step S140 in the middle; Figure 3 yes Figure 1 The flowchart of step S150 in the middle; Figure 4 yes Figure 3 The flowchart of step S310 in the process; Figure 5 yes Figure 3 The flowchart of step S330 in the text; Figure 6 yes Figure 3 The flowchart of step S340 in the text; Figure 7 This is another flowchart of the dialogue-based intent recognition method provided in the embodiments of this application; Figure 8This is a schematic diagram of the structure of the dialogue-based intent recognition device provided in the embodiments of this application; Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0018] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0020] During service interactions, identifying user intent allows for the optimization of service strategies for agents, thereby improving business service conversion rates. Taking insurance purchases in a fintech scenario as an example, if the insurance service system detects low customer intent to purchase insurance, it can immediately push incentive tools such as premium coupons and limited-time free medical examinations to service agents, who can then deliver these to customers in real time to increase the insurance purchase conversion rate. Similarly, in a healthcare scenario, for online follow-up consultations, if the medical service system detects low patient intent to return for a follow-up visit, it can immediately push patient service strategies to service agents, enabling them to offer services such as medical insurance reimbursement and weekend queue-free appointments, thereby improving the follow-up consultation conversion rate.

[0021] In related technologies, keyword matching and sentiment polarity judgment are commonly used for intent recognition. However, in service interactions, users often do not directly express their true intentions. These methods can only capture the literal meaning of the dialogue and cannot deeply understand the conversation between the user and the service agent, resulting in low accuracy of intent recognition.

[0022] Based on this, embodiments of this application provide a dialogue-based intent recognition method, a dialogue-based intent recognition device, an electronic device, and a computer-readable storage medium, aiming to improve the accuracy of intent recognition.

[0023] The dialogue-based intent recognition method, device, electronic device, and computer-readable storage medium provided in this application are specifically described through the following embodiments. First, the dialogue-based intent recognition method in this application is described.

[0024] The dialogue-based intent recognition method provided in this application relates to the field of artificial intelligence technology. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application implementing the dialogue-based intent recognition method, but is not limited to the above forms.

[0025] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0026] Figure 1 This is an optional flowchart of the dialogue-based intent recognition method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S110 to S150.

[0027] Step S110: Obtain the voice dialogue between the target object and the service agent; wherein the voice dialogue includes at least one voice segment. Step S120: Extract speech features from the speech segment to obtain speech prosody features and temporal behavior features; Step S130: Perform speech recognition on the speech segment to obtain the dialogue text, and extract text features from the dialogue text to obtain text semantic features. Step S140: Attention features are extracted based on speech prosody features, temporal behavior features, text semantic features and preset historical dialogue features to obtain contextual attention features; wherein, historical dialogue features are obtained by extracting features from the historical dialogue voice between the target object and the service agent, and the dialogue end time of the historical dialogue voice is before the dialogue start time of the dialogue voice. Step S150: Perform intent recognition on the target object based on speech prosody features, temporal behavior features, text semantic features, and contextual attention features.

[0028] Steps S110 to S150 as shown in the embodiments of this application, by combining multimodal features such as speech prosody features, temporal behavior features, text semantic features and contextual attention features for intent recognition, can better understand the deep semantics hidden in the dialogue speech, thereby improving the accuracy of intent recognition.

[0029] In step S110 of some embodiments, the service agent refers to an interactive unit that provides business services to users. It can be a physical agent or a virtual service channel. The target object is the end user who needs business services. Business services can be provided to the target object through the service agent. The dialogue voice between the target object and the service agent is acquired to identify the target object's intent based on the dialogue voice, and the service agent's service strategy is optimized according to the intent, thereby improving the service quality of the service agent. The dialogue voice includes at least one voice segment. The target object may have multiple rounds of dialogue with the service agent. To avoid confusion of intent across rounds, the dialogue voice can be divided into voice segments according to the rounds. At this time, one voice segment corresponds to one round of dialogue. The dialogue voice of a single round may be long. Long voice has the characteristics of complexity, redundancy, and mixed information. In order to reduce computational complexity and improve feature quality, the dialogue voice can be segmented into voice segments.

[0030] In step S120 of some embodiments, speech features are extracted from the speech segment using a speech encoder to obtain speech prosodic features. The speech encoder converts the speech signal into a feature vector that accurately represents prosodic information. The speech encoder can employ a transformer model, convolutional neural network, etc. Speech prosodic features reflect the rhythm, pitch, strength, and speed of the speech signal, and are an important form of language and emotional expression, including features such as fundamental frequency, energy, speech rate, intonation, pause frequency, and pause duration. Speech prosodic features allow for objective and quantitative assessment of the service level of agents and can capture the implicit states and needs of the target audience, such as emotions, service urgency, and satisfaction. Temporal behavior features are obtained by analyzing the temporal patterns of the speech segment using a temporal encoder. A temporal encoder is a model that compresses signals that change over time into useful vectors, such as long short-term memory networks and temporal convolutional networks. Temporal behavior features reflect the characteristics of the target audience and service agents' language behavior patterns in the time dimension, including the object of the language behavior, the time of occurrence, the temporal correlation of the language behavior, and the change pattern. By exploring time-related behavioral patterns, we can reflect the service strategies, conversational skills, and implicit states and needs of the target audience. The target audience's intentions change with the service strategies of the service agents. Through the two-way rhythmic and temporal behavioral linkage between the service agents and the target audience, we can more accurately identify the target audience's intentions.

[0031] In step S130 of some embodiments, a speech recognition model is used to perform speech recognition on the speech segment, converting the speech signal into text to obtain dialogue text. Then, a text encoder is used to extract text features from the dialogue text to extract deep semantic information and obtain text semantic features. The text encoder can be a BERT model.

[0032] The target's intent is not only directly related to the current dialogue content but also relies on the support of the dialogue context. Furthermore, the target's intent may change depending on the service agent's service strategy, requiring the capture of its true intent through context. To achieve more accurate intent recognition, this application introduces historical dialogue features to recall deep context, and extracts attention features from speech prosody features, temporal behavior features, text semantic features, and historical dialogue features based on an attention mechanism to obtain contextual attention features, thereby achieving a leap from literal matching to contextual understanding in intent recognition. Historical dialogue features are obtained by extracting features from the historical dialogue between the target and the service agent, where the dialogue end time is before the dialogue start time.

[0033] Historical dialogue speech includes at least one historical speech segment, and historical dialogue features are a set of features of the segment dialogue features of the historical speech segment. The feature extraction process of historical dialogue features is as follows: referring to step S120, speech features are extracted from the historical speech segment to obtain segment prosodic features and segment temporal behavior features; referring to step S130, speech recognition is performed on the historical speech segment to obtain historical dialogue text, and text features are extracted from the historical dialogue text to obtain segment text semantic features; the segment prosodic features, segment temporal behavior features, and segment text semantic features are concatenated to obtain the segment dialogue features of the historical speech segment. The segment dialogue features of each historical speech segment are concatenated to obtain the historical dialogue features.

[0034] Historical dialogue features include historical prosodic features, historical temporal features, and historical text features. Historical prosodic features are a set of segment prosodic features for each historical speech segment, historical temporal features are a set of segment temporal behavioral features for each historical speech segment, and historical text features are a set of segment text semantic features for each historical speech segment.

[0035] Please see Figure 2 In some embodiments, step S140 may include, but is not limited to, steps S210 to S240: Step S210: Calculate the query vector based on the prosodic features, temporal behavior features, and text semantic features of each speech segment; Step S220: Calculate the key vector and value vector based on the features of the dialogue segments; Step S230: Perform attention calculation based on the query vector, key vector, and value vector to obtain segment attention features; Step S240: Perform feature fusion on the segment attention features to obtain context attention features.

[0036] In step S210 of some embodiments, to filter out the contextual information most relevant to the current intent from historical dialogue speech, this application embodiment is based on an attention mechanism. It calculates a query vector based on features extracted from the current dialogue speech and calculates key and value vectors based on features extracted from historical dialogue speech. Specifically, the prosodic features, temporal behavior features, and textual semantic features of each speech segment in the dialogue speech are concatenated or fused to obtain intermediate features. The query weight matrix of the attention mechanism is multiplied by the intermediate features to obtain the query vector.

[0037] In step S220 of some embodiments, for each historical speech segment, the key weight matrix of the attention mechanism is multiplied by the segment dialogue features to obtain a key vector, and the value weight matrix of the attention mechanism is multiplied by the segment dialogue features to obtain a value vector.

[0038] In step S230 of some embodiments, attention is calculated based on the query vector, key vector, and value vector corresponding to the historical speech segment to focus on historical key information associated with the current dialogue speech, thereby obtaining the segment attention features of the historical speech segment. The formula for attention calculation is expressed as: , Where attention represents the segment attention feature; K and V represent the query vector, key vector, and value vector, respectively; This represents the dimension of the key vector.

[0039] In step S240 of some embodiments, the segment attention features of different historical speech segments focus on features of different dimensions. The features focused by a single segment attention feature are relatively one-sided and cannot cover the global context. To improve the accuracy of intent recognition, this application embodiment performs feature fusion on the segment attention features of each historical speech segment to upgrade the local key information of each segment into contextual attention features that cover the whole and focus on the core, thereby improving the depth of understanding of the dialogue context. Feature fusion methods can include feature addition, weighted average fusion, temporal convolution fusion, etc.

[0040] Through the above steps S210 to S240, the local correlation information of each historical speech segment is preserved, and the temporal logic and global correlation between each historical speech segment are captured, resulting in a unified and comprehensive contextual attention feature, thereby providing complete contextual support for intent recognition.

[0041] Please see Figure 3 In some embodiments, step S150 may include, but is not limited to, steps S310 to S340: Step S310: For each speech segment, the speech prosodic features, temporal behavior features and text semantic features are weighted and fused to obtain the segment fusion features; Step S320: Perform feature fusion on the fusion features of each segment to obtain the first fusion feature; Step S330: Perform weighted fusion of the first fusion feature and the contextual attention feature to obtain the second fusion feature; Step S340: Perform intent recognition on the target object based on the second fusion feature.

[0042] In step S310 of some embodiments, for each speech segment, feature weight parameters are determined for the speech prosody features, temporal behavior features, and text semantic features corresponding to the speech segment. The speech prosody features, temporal behavior features, and text semantic features are then weighted and fused according to the feature weight parameters to obtain segment fusion features, thereby accurately extracting the semantic features of the speech segment. The feature weight parameters are used to indicate the importance of the features to the intent recognition task.

[0043] The segment fusion feature of the i-th speech segment is represented as: , in, This represents the segment fusion feature of the i-th speech segment; and Let represent the text semantic features and their feature weight parameters of the i-th speech segment, respectively; and Let represent the prosodic features of the i-th speech segment and their feature weight parameters, respectively; and Let represent the temporal behavior features of the i-th speech segment and its feature weight parameters, respectively.

[0044] In step S320 of some embodiments, the overall semantics of the dialogue speech is closely related to the semantics of each speech segment. To accurately understand the complete semantics of the dialogue speech, the segment fusion features of each speech segment are added together, and the average of the added features is calculated based on the number of speech segments to aggregate the semantic information of each speech segment, forming a coherent global semantic expression, thus obtaining the first fusion feature. The first fusion feature is represented as: , in, The first fusion feature is represented by T; T represents the number of speech segments. This represents the segment fusion feature of the i-th speech segment.

[0045] In step S330 of some embodiments, the first fusion feature is a global feature of the dialogue speech, which can ensure that semantic understanding does not deviate from the overall context. The context attention feature focuses on the correlation between the current dialogue speech and the historical dialogue speech, which can capture temporal logic. By weighted fusion of the first fusion feature and the context attention feature, a second fusion feature is obtained, which can achieve deep semantic understanding of the dialogue speech. Specifically, the feature weight parameter of the first fusion feature is determined and multiplied by the first fusion feature. The feature weight parameter of the context attention feature is determined and multiplied by the context attention feature. The results of these two multiplications are added together to obtain the second fusion feature. The second fusion feature is represented as: , in, and These represent the first fusion feature and its feature weight parameters, respectively. and These represent the context attention features and their feature weight parameters, respectively.

[0046] In step S340 of some embodiments, the second fused feature is input into the intent recognition model to identify the intent of the target object, thereby obtaining the intent category of the target object and the intent score of the intent category. The intent score reflects the service agent's probability of facilitating the service; the higher the intent score, the higher the probability of the service agent facilitating the service. For example, if the target object's intent category is insurance purchase and the intent score is 0.8, it indicates that the service agent has a high probability of facilitating the target object's insurance purchase.

[0047] Steps S310 to S340 above, through multi-feature fusion, can achieve deep semantic understanding of dialogue speech, thereby improving the accuracy of intent recognition.

[0048] Please see Figure 4 In some embodiments, step S310 may include, but is not limited to, steps S410 to S440: Step S410: Attention is calculated based on historical prosodic features and phonological prosodic features to obtain prosodic weights, and the prosodic weights are multiplied by the phonological prosodic features to obtain weighted prosodic features. Step S420: Calculate attention based on historical time series features and time series behavioral features to obtain time series weights, and multiply the time series weights and time series behavioral features to obtain weighted time series features; Step S430: Attention is calculated based on historical text features and text semantic features to obtain semantic weights, and the semantic weights are multiplied by the text semantic features to obtain weighted text features; Step S440: The weighted prosodic features, weighted temporal features, and weighted text features are summed to obtain the fragment fusion features.

[0049] In step S410 of some embodiments, the importance of each feature varies depending on the intent recognition scenario. For example, textual semantic features are more important when interpreting insurance terms; speech prosody features are more important when perceiving user emotions. To achieve adaptive feature fusion and improve the accuracy of feature fusion, this embodiment uses an attention mechanism to dynamically calculate the feature weight parameters of speech prosody features, temporal behavior features, and textual semantic features, instead of using fixed values, so that the importance of different features can be adaptively adjusted according to different dialogue contexts. It should be noted that this attention mechanism is a separate mechanism from the attention mechanism in step S210. Specifically, the speech prosody features are multiplied by the query weight of the attention mechanism to obtain a query vector. The historical prosody features are multiplied by the key weight of the attention mechanism to obtain a key vector. The historical prosody features are multiplied by the value weight of the attention mechanism to obtain a value vector. Referring to step S230, attention is calculated based on the query vector, key vector, and value vector to obtain the prosodic weight. The prosodic weight is the feature weight parameter of the speech prosody features. The prosodic weight is multiplied by the speech prosody features to obtain the weighted prosodic features.

[0050] In step S420 of some embodiments, the temporal behavior features and the query weights of the attention mechanism are multiplied to obtain a query vector. The historical temporal features and the key weights of the attention mechanism are multiplied to obtain a key vector. The historical temporal features and the value weights of the attention mechanism are multiplied to obtain a value vector. Referring to step S230, attention is calculated based on the query vector, key vector, and value vector to obtain temporal weights. The temporal weights are the feature weight parameters of the temporal behavior features. The temporal weights and temporal behavior features are multiplied to obtain weighted temporal features.

[0051] In step S430 of some embodiments, the text semantic features and the query weights of the attention mechanism are multiplied to obtain a query vector. The historical text features and the key weights of the attention mechanism are multiplied to obtain a key vector. The historical text features and the value weights of the attention mechanism are multiplied to obtain a value vector. Referring to step S230, attention is calculated based on the query vector, key vector, and value vector to obtain semantic weights. The semantic weights are the feature weight parameters of the text semantic features. The semantic weights are multiplied by the text semantic features to obtain weighted text features.

[0052] In step S440 of some embodiments, the weighted prosodic features, weighted temporal features, and weighted text features are added together to obtain fragment fusion features.

[0053] Through steps S410 to S440, the feature weight parameters can be adaptively adjusted according to the dialogue context, thereby improving the accuracy of feature fusion.

[0054] Please see Figure 5In some embodiments, step S330 may include, but is not limited to, steps S510 to S540: Step S510: Calculate the fusion weight of the first fusion feature based on the speech prosody features, temporal behavior features, text semantic features and context attention features, and multiply the fusion weight by the first fusion feature to obtain the weighted fusion feature; Step S520: Obtain the dialogue stage of the conversation voice; Step S530: Calculate the attention weights of the contextual attention features based on the dialogue stage, and multiply the attention weights by the contextual attention features to obtain the weighted attention features; Step S540: Add the weighted fusion features and the weighted attention features together to obtain the second fusion feature.

[0055] In step S510 of some embodiments, speech prosody features, temporal behavior features, text semantic features, and contextual attention features are input into a lightweight neural network. The feature weight parameters of the first fused feature are dynamically predicted by the lightweight neural network to obtain the fusion weight, enabling the feature weight of the first fused feature to be dynamically adjusted according to the dialogue context. The network structure of the lightweight neural network can be designed according to the actual situation. For example, the lightweight neural network includes an input layer, a fully connected layer, an activation layer, and an output layer. Existing lightweight neural networks, such as MobileNet, can also be used. The fusion weight is multiplied by the first fused feature to obtain the weighted fused feature.

[0056] In step S520 of some embodiments, the voice dialogue between the target object and the service agent is input into a classifier, which determines the dialogue stage of the voice dialogue. The dialogue stage refers to the business stage to which the voice dialogue belongs, including the opening stage, demand mining stage, quotation stage, or transaction stage, etc.

[0057] In step S530 of some embodiments, the influence of contextual information on intent recognition varies at different dialogue stages. For example, in the opening stage, contextual information has a smaller impact on intent recognition; in the closing stage, the continuity of the dialogue is emphasized, and contextual information has a larger impact on intent recognition. This application embodiment dynamically adjusts the influence of contextual information on intent recognition according to the dialogue stage. Specifically, attention weights of contextual attention features are selected from a preset mapping table based on the dialogue stage, so that the attention weights change with the dialogue stage. The preset mapping table indicates the mapping relationship between the dialogue stage and the attention weights. For example, the attention weight in the opening stage is 0.1, and the attention weight in the closing stage is 0.9. The attention weights and contextual attention features are multiplied to obtain weighted attention features.

[0058] In step S540 of some embodiments, the weighted fusion feature and the weighted attention feature are added together to obtain the second fusion feature.

[0059] Through the above steps S510 to S540, the accuracy of feature fusion is improved while ensuring adaptive feature fusion.

[0060] Please see Figure 6 In some embodiments, step S340 may include, but is not limited to, steps S610 to S620: Step S610: Extract spatiotemporal features from the second fusion feature to obtain spatiotemporal attention features; Step S620: Perform intent recognition on the target object based on spatiotemporal attention characteristics.

[0061] In step S610 of some embodiments, an attention mechanism is used to perform attention calculation on the second fused feature in the temporal dimension to obtain a temporal attention feature. An attention mechanism is also used to perform attention calculation on the second fused feature in the spatial dimension to obtain a spatial attention feature. The temporal attention feature and the spatial attention feature are then fused to obtain a spatiotemporal attention feature.

[0062] In step S620 of some embodiments, spatiotemporal attention features are input into the intent recognition model to perform intent recognition on the target object, and intent category and intent score of intent category are obtained.

[0063] Through the above steps S610 to S620, it is possible to simultaneously capture the spatial dimension of correlation and the temporal dimension of dependency, so that when recognizing intent, it can focus on local key information and track the dynamic changes in time sequence, thereby improving the accuracy of intent recognition.

[0064] Please see Figure 7 In some embodiments, after step S150, the dialogue-based intent recognition method may also include, but is not limited to, steps S710 to S740: Step S710: Obtain the intent category obtained by performing intent recognition on the target object; Step S720: Perform sentiment analysis based on speech prosody features and text semantic features to obtain the sentiment category of the target object; Step S730: Select target scripts for service agents from the preset script library based on emotion category and intent category; Step S740: In response to receiving the target object's feedback information on the target script, generate a follow-up script for the service agent based on the feedback information.

[0065] In step S710 of some embodiments, the intent category obtained by performing intent recognition on the target object is acquired, and the intent category has an intent score.

[0066] In step S720 of some embodiments, during the service provision process, the service agent needs to pay attention to the emotional changes of the target audience in order to adjust the service strategy in real time according to the emotional changes. Specifically, an emotional classification model is used to perform emotional analysis on the prosodic features of speech and the semantic features of text to obtain the emotional category of the target audience. The emotional category can be positive emotions such as joy and satisfaction, or negative emotions such as anger, anxiety, and aversion.

[0067] In step S730 of some embodiments, a scoring threshold is set, such as 0.7. If the intent score of the intent category is greater than the scoring threshold, it indicates that the service agent has a high probability of facilitating the service and the emotion category is positive emotion. Then, the agent retrieves the dialogue scripts similar to the current dialogue from the preset script library to obtain the target scripts for the service agent.

[0068] The pre-set script library contains sample scripts that have successfully facilitated services. The script vectors of these sample scripts can be extracted, and their features—including prosodic features, temporal behavioral features, textual semantic features, and contextual attention features—can be concatenated or fused to obtain dialogue vectors. The similarity between the script vectors and the dialogue vectors is calculated, and the top K most similar sample scripts are selected as the target scripts for service agents.

[0069] If the intent score for the intent category is less than or equal to the scoring threshold, or if the sentiment category is negative, then the current dialogue features, historical dialogue features, intent category and its intent score, and sentiment category of the spoken dialogue are input into the large language model. Leveraging the powerful semantic understanding capabilities of the large language model, the target script for the service agent is generated. Current dialogue features include prosodic features, temporal behavioral features, and textual semantic features.

[0070] In step S740 of some embodiments, the scripted feedback information is used to indicate whether the target object has performed a transaction. In response to receiving scripted feedback information from the target object regarding the target script, if the scripted feedback information indicates that the target object has not performed a transaction, it means the service agent needs to continue following up with the target object. The current dialogue features and historical dialogue features of the spoken dialogue can be input into the large language model to generate a follow-up script for the service agent. If the scripted feedback information indicates that the target object has performed a transaction, it means the service agent does not need to follow up with the target object.

[0071] Through the above steps S710 to S740, the service quality of service agents can be improved.

[0072] Please see Figure 8This application also provides a dialogue-based intent recognition device that can implement the above-described dialogue-based intent recognition method. The dialogue-based intent recognition device includes: The voice acquisition module 810 is used to acquire the voice dialogue between the target object and the service agent; wherein the voice dialogue includes at least one voice segment. The speech feature extraction module 820 is used to extract speech features from speech segments to obtain speech prosody features and temporal behavior features; The text feature extraction module 830 is used to perform speech recognition on speech segments to obtain dialogue text, and to extract text features from the dialogue text to obtain text semantic features. The attention feature extraction module 840 is used to extract attention features based on speech prosody features, temporal behavior features, text semantic features and preset historical dialogue features to obtain contextual attention features; wherein, the historical dialogue features are obtained by extracting features from the historical dialogue voice between the target object and the service agent, and the dialogue end time of the historical dialogue voice is before the dialogue start time of the dialogue voice. The intent recognition module 850 is used to recognize the intent of a target object based on speech prosody features, temporal behavior features, text semantic features, and contextual attention features.

[0073] The specific implementation of this dialogue-based intent recognition device is basically the same as the specific implementation of the dialogue-based intent recognition method described above, and will not be repeated here.

[0074] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described dialogue-based intent recognition method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0075] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 910 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 920 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 920 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 920 and is called and executed by the processor 910 to execute the dialogue-based intent recognition method of the embodiments of this application. The input / output interface 930 is used to implement information input and output; The communication interface 940 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 950 transmits information between various components of the device (e.g., processor 910, memory 920, input / output interface 930, and communication interface 940); The processor 910, memory 920, input / output interface 930 and communication interface 940 are connected to each other within the device via bus 950.

[0076] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described dialogue-based intent recognition method.

[0077] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0078] The dialogue-based intent recognition method, device, electronic device, and computer storage medium provided in this application, by combining multimodal features such as speech prosody features, temporal behavior features, text semantic features, and contextual attention features for intent recognition, can better understand the deep semantics hidden in dialogue speech, thereby improving the accuracy of intent recognition.

[0079] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0080] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0081] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0082] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0083] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0084] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0085] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0086] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0087] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A dialog-based intent recognition method, characterized by, The method comprises: acquiring conversation voice between a target object and a service agent; wherein the conversation voice comprises at least one voice segment; extracting voice features from the voice segment to obtain voice prosody features and timing behavior features; performing voice recognition on the voice segment to obtain conversation text, and extracting text features from the conversation text to obtain text semantic features; extracting attention features according to the voice prosody features, the timing behavior features, the text semantic features and preset historical conversation features to obtain context attention features; wherein the historical conversation features are obtained by feature extraction on historical conversation voice between the target object and the service agent, and the conversation end time of the historical conversation voice is before the conversation start time of the conversation voice; performing intent recognition on the target object according to the voice prosody features, the timing behavior features, the text semantic features and the context attention features.

2. The method of claim 1, wherein, The historical conversation voice comprises historical voice segments, and the historical conversation features comprise segment conversation features of the historical voice segments. The extracting attention features according to the voice prosody features, the timing behavior features, the text semantic features and preset historical conversation features to obtain context attention features comprises: calculating query vectors according to the voice prosody features, the timing behavior features and the text semantic features of each voice segment; calculating key vectors and value vectors according to the segment conversation features; performing attention calculation according to the query vectors, the key vectors and the value vectors to obtain segment attention features; performing feature fusion on the segment attention features to obtain the context attention features.

3. The method of claim 1, wherein, The performing intent recognition on the target object according to the voice prosody features, the timing behavior features, the text semantic features and the context attention features comprises: performing weighted fusion on the voice prosody features, the timing behavior features and the text semantic features to obtain segment fusion features for each voice segment; performing feature fusion on each segment fusion feature to obtain a first fusion feature; performing weighted fusion on the first fusion feature and the context attention feature to obtain a second fusion feature; performing intent recognition on the target object according to the second fusion feature.

4. The method of claim 3, wherein, The historical conversation features comprise historical prosody features, historical timing features and historical text features. The performing weighted fusion on the voice prosody features, the timing behavior features and the text semantic features to obtain segment fusion features comprises: performing attention calculation according to the historical prosody features and the voice prosody features to obtain prosody weights, and multiplying the prosody weights and the voice prosody features to obtain weighted prosody features; performing attention calculation according to the historical timing features and the timing behavior features to obtain timing weights, and multiplying the timing weights and the timing behavior features to obtain weighted timing features; According to the historical text features and the text semantic features, attention calculation is performed to obtain a semantic weight, and the semantic weight and the text semantic features are multiplied to obtain weighted text features; The weighted prosody features, the weighted time sequence features and the weighted text features are added to obtain the segment fusion features.

5. The method of claim 3, wherein, The first fusion features and the context attention features are weighted and fused to obtain second fusion features, including: According to the speech prosody features, the time sequence behavior features, the text semantic features and the context attention features, a fusion weight of the first fusion features is calculated, and the fusion weight and the first fusion features are multiplied to obtain weighted fusion features; A dialogue stage of the dialogue speech is obtained; According to the dialogue stage, an attention weight of the context attention features is calculated, and the attention weight and the context attention features are multiplied to obtain weighted attention features; The weighted fusion features and the weighted attention features are added to obtain the second fusion features.

6. The method of claim 3, wherein, According to the second fusion features, intent recognition is performed on the target object, including: Temporal and spatial features are extracted from the second fusion features to obtain temporal and spatial attention features; According to the temporal and spatial attention features, intent recognition is performed on the target object.

7. The method according to any one of claims 1 to 6, characterized in that, After the target object is recognized according to the speech prosody features, the time sequence behavior features, the text semantic features and the context attention features, the method further includes: An intent category obtained by performing intent recognition on the target object is obtained; According to the speech prosody features and the text semantic features, sentiment analysis is performed to obtain a sentiment category of the target object; According to the sentiment category and the intent category, a target script of the service agent is filtered from a preset script library; In response to receiving script feedback information of the target object on the target script, follow-up script of the service agent is generated according to the script feedback information.

8. A dialogue-based intent recognition apparatus, characterized by comprising: The device includes: A speech acquisition module is configured to acquire dialogue speech between a target object and a service agent; wherein the dialogue speech includes at least one speech segment; A speech feature extraction module is configured to perform speech feature extraction on the speech segment to obtain speech prosody features and time sequence behavior features; A text feature extraction module is configured to perform speech recognition on the speech segment to obtain dialogue text, and perform text feature extraction on the dialogue text to obtain text semantic features; An attention feature extraction module is configured to perform attention feature extraction on the speech prosody features, the time sequence behavior features, the text semantic features and preset historical dialogue features to obtain context attention features; wherein the historical dialogue features are obtained by performing feature extraction on historical dialogue speech between the target object and the service agent, and a dialogue end time of the historical dialogue speech is before a dialogue start time of the dialogue speech. An intent recognition module is configured to perform intent recognition on the target object according to the speech prosody feature, the timing behavior feature, the text semantic feature, and the context attention feature.

9. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the method in any one of claims 1 to 7.

Images