Label generation method and device based on emotion recognition, equipment and medium

Abstract

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as financial science and technology and medical health, and discloses a label generation method and device based on emotion recognition, equipment and a medium, and the method comprises the steps: carrying out the preprocessing of a target audio, and obtaining the processed audio data; executing voice transcription based on the processed audio data and extracting acoustic features; performing emotion analysis on the acoustic features and the transliterated text; executing intention and appeal recognition based on the transliteration text; performing enhanced matching on the emotion analysis result and the intention and appeal recognition result in a knowledge base; and generating a label set according to a knowledge base enhancement result. According to the method, the acoustic and semantic information of the audio data is jointly processed, and the knowledge base is combined for enhancing matching to realize multi-dimensional tag generation, so that the recognition accuracy and the industry adaptability are improved, and the structured application of the audio data is supported.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, device, and medium for generating labels based on emotion recognition. Background Technology

[0002] In the fintech sector, insurance companies, banks, and related institutions need to process a large amount of audio data related to customer reports, inquiries, claims, and business transactions. Existing speech recognition technologies primarily rely on general automatic speech recognition models to transcribe audio into text, but their accuracy is often insufficient when dealing with specialized financial terminology. For example, key terms such as "critical illness insurance," "deductible," and "subrogation" are easily misidentified or missed, resulting in the inability to fully convert crucial customer information into usable structured data. Simultaneously, existing emotion recognition methods largely depend on changes in single acoustic features, such as volume, tone, or speech rate. This approach struggles to accurately capture implicit dissatisfaction in calm expressions or potential complaints in a composed tone, thus affecting the business department's accurate assessment of customer risk and urgency. Furthermore, existing intent recognition models are mostly geared towards general customer service scenarios and struggle to accurately extract business requests specific to the financial sector, such as claims progress inquiries and fund transfer status confirmations. This prevents audio content from being effectively integrated with financial knowledge bases, limiting the depth of automated processing and business collaboration. Furthermore, current tag generation is usually limited to simple emotion tags or call category tags, lacking refined output on urgency and sensitivity levels. This hinders the implementation of financial scenarios such as claims priority allocation, customer service quality inspection, and risk warning.

[0003] In the healthcare sector, medical institutions and health service platforms also require intelligent processing of audio data from patient consultations, diagnoses, and health follow-ups. General speech recognition methods fall short when dealing with medical terminology; terms like "allergic purpura," "drug dosage," and "imaging examination report" are easily misidentified, leading to distortion of information in structured doctor-patient communication. Existing emotion recognition methods also primarily rely on acoustic features, neglecting the role of textual semantic features in emotional expression, making it difficult to capture anxiety or dissatisfaction masked by a patient's calm tone. Existing intent recognition models are mostly used in general dialogue scenarios, struggling to accurately extract specific medical needs, such as "follow-up appointment" and "drug side effect feedback," and lack effective linkage mechanisms with medical knowledge bases, failing to achieve direct mapping from audio data to treatment processes. Furthermore, the lack of urgency and sensitivity dimensions in tagged outputs hinders medical institutions from prioritizing triage, issuing health risk warnings, and monitoring service quality based on patient needs and emotional states, limiting the business value conversion of medical audio data. Summary of the Invention

[0004] The main objective of this invention is to provide a method, apparatus, device, and storage medium for generating tags based on emotion recognition, aiming to solve the technical problem that existing technologies cannot comprehensively identify and form structured tags for emotions, intentions, and demands in audio data, resulting in deficiencies in industry adaptability and tagged output.

[0005] To achieve the above objectives, the present invention provides a label generation method based on emotion recognition, comprising:

[0006] The target audio is acquired and audio preprocessing is performed to obtain the processed audio data;

[0007] Based on the processed audio data, a speech-to-text operation is performed to generate transcribed text, and acoustic features are extracted based on the processed audio data.

[0008] Sentiment analysis is performed on the acoustic features and the transcribed text to generate sentiment analysis results.

[0009] Based on the transcribed text, the intent and appeal are identified and a set of elements are extracted to generate the intent and appeal identification result.

[0010] The emotion analysis results and the intent and appeal recognition results are subjected to enhanced matching within the knowledge base to obtain the knowledge base enhanced results.

[0011] Based on the enhanced results of the knowledge base, a set of tags is generated, including call type, core demand, emotional state, urgency level, and sensitivity level.

[0012] Furthermore, to achieve the above objectives, the present invention provides a tag generation device based on emotion recognition, comprising:

[0013] The audio preprocessing module is used to acquire the target audio and perform audio preprocessing to obtain the processed audio data;

[0014] The speech transcription and feature extraction module is used to perform speech transcription operations based on the processed audio data, generate transcribed text, and extract acoustic features based on the processed audio data.

[0015] The sentiment analysis module is used to perform sentiment analysis on the acoustic features and the transcribed text, and generate sentiment analysis results.

[0016] The intent recognition module is used to perform intent and appeal recognition based on the transcribed text, extract a set of elements, and generate intent and appeal recognition results;

[0017] The knowledge base enhancement matching module is used to perform enhancement matching between the sentiment analysis results and the intent and appeal recognition results within the knowledge base to obtain the knowledge base enhancement results;

[0018] The tag generation module is used to generate a set of tags based on the knowledge base enhancement results, including call type, core demand, emotional state, urgency level, and sensitivity level.

[0019] Furthermore, to achieve the above objectives, the present invention also provides a computer device, the computer device including a memory, a processor, and an emotion recognition-based tag generation program stored in the memory and executable on the processor, wherein when the emotion recognition-based tag generation program is executed by the processor, it implements the steps of the emotion recognition-based tag generation method as described above.

[0020] Furthermore, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an emotion-based tag generation program, wherein the emotion-based tag generation program, when executed by a processor, implements the steps of the emotion-based tag generation method as described above.

[0021] Beneficial Effects: This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses a method, apparatus, device, and medium for generating tags based on emotion recognition, comprising: acquiring target audio and performing audio preprocessing to obtain processed audio data; performing speech-to-text transcription based on the processed audio data to generate transcribed text and extracting acoustic features; performing emotion analysis on the acoustic features and transcribed text to generate emotion analysis results; performing intent and appeal recognition based on the transcribed text and extracting a set of elements to generate intent and appeal recognition results; performing enhanced matching on the emotion analysis results and intent and appeal recognition results within a knowledge base to obtain knowledge base enhanced results; and generating a tag set based on the knowledge base enhanced results. This invention achieves a synergistic improvement in emotion analysis and intent recognition by jointly processing the acoustic features of audio data and the transcribed text, and generates multi-dimensional tags by combining knowledge base enhanced matching. This not only improves the accuracy and industry adaptability of audio recognition during the audio understanding process but also forms structured tag outputs that can be directly used in business applications, enhancing the value conversion capability of audio data. Attached Figure Description

[0022] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0023] Figure 1 This is a schematic diagram of an application environment for a label generation method based on emotion recognition in one embodiment of the present invention;

[0024] Figure 2 This is a flowchart illustrating an embodiment of the label generation method based on emotion recognition of the present invention;

[0025] Figure 3This is a schematic diagram of the functional modules of a preferred embodiment of the emotion recognition-based tag generation device of the present invention;

[0026] Figure 4 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0027] Figure 5 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0028] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0029] The emotion recognition-based tag generation method provided in this invention can be applied to, for example... Figure 1 In this application environment, the client communicates with the server via a network. The server can acquire the target audio from the client and perform audio preprocessing to obtain processed audio data; based on the processed audio data, it performs speech-to-text transcription to generate transcribed text and extracts acoustic features; it performs sentiment analysis on the acoustic features and transcribed text to generate sentiment analysis results; based on the transcribed text, it performs intent and appeal recognition and extracts a set of elements to generate intent and appeal recognition results; it performs enhanced matching on the sentiment analysis results and intent and appeal recognition results within a knowledge base to obtain knowledge base enhanced results; and it generates a tag set based on the knowledge base enhanced results. This invention achieves synergistic improvement in sentiment analysis and intent recognition by jointly processing the acoustic features of audio data and transcribed text, and generates multi-dimensional tags by combining knowledge base enhanced matching. This not only improves the accuracy and industry adaptability of audio recognition during the audio understanding process, but also forms structured tag outputs that can be directly used in business applications, enhancing the value conversion capability of audio data. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The present invention will now be described in detail through specific embodiments.

[0030] Please see Figure 2 , Figure 2 This is a flowchart illustrating an embodiment of the emotion recognition-based tag generation method provided by the present invention. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0031] like Figure 2 As shown, the label generation method based on emotion recognition proposed in this invention includes the following steps:

[0032] S10: Acquire the target audio and perform audio preprocessing to obtain the processed audio data;

[0033] In this embodiment, audio acquisition involves capturing the raw sound source and converting it into a digital signal that can be processed computationally. This process typically relies on a microphone array or the audio input module of a terminal device. In implementation, the acquisition needs to ensure that the sampling rate and quantization accuracy meet the requirements of subsequent processing, for example, using a sampling rate of 16kHz or higher to ensure complete preservation of speech features. The audio acquisition targets not only the user's continuous speech during a call but may also include complex mixed signals with environmental noise, background noise, or even multiple speakers. These inputs determine the difficulty of adapting the subsequent processing modules.

[0034] Audio preprocessing is the process of cleaning and standardizing the acquired raw audio signal. The first task of preprocessing is noise suppression, which typically employs spectral subtraction, Wiener filtering, or adaptive denoising models trained with deep neural networks. This preserves the effective information of the speech by analyzing the noise spectrum and reducing its energy. Next is speaker segmentation, used to divide the audio stream into independent trajectories when multiple speakers are speaking simultaneously. Common methods include clustering algorithms based on Hidden Markov Models and source separation methods based on end-to-end deep networks. After segmentation, further speech segmentation is required to divide long audio segments into shorter, more easily analyzable segments. Segmentation criteria can be based on silence detection, energy thresholding, or endpoint detection algorithms.

[0035] After segmentation, audio quality testing is required. This step involves evaluating the audio's clarity, signal-to-noise ratio, and integrity, for example, by calculating short-time energy and zero crossover rate to determine if the audio is distorted. Audio segments that pass the testing are assigned segment tags for indexing and tracing in subsequent processing. This step ensures that the processed audio data is a structured and traceable input, providing a standardized data foundation for subsequent speech transcription and feature extraction.

[0036] In the implementation process, different noise suppression algorithms can be employed. For example, in call center scenarios with stable voice environments, frequency domain filtering methods can be used to quickly suppress constant background noise. In mobile applications with complex and variable environments, adaptive denoising models trained with convolutional neural networks or recurrent neural networks can be used to improve denoising performance. Speaker separation can be based on traditional Gaussian mixture model clustering or the deep embedding clustering method proposed in recent years, which can significantly improve separation accuracy under multi-channel audio input. For speech segmentation, a silence detection threshold can be dynamically adjusted to accurately identify segment boundaries even when the user pauses for a long time, avoiding the loss of semantic information due to premature cutoff of silence. In audio quality detection, a quality prediction model based on convolutional networks can be introduced to achieve rapid detection of blurry, distorted, and over-compressed audio. Furthermore, the confidence level of segment labels can be dynamically assigned during the detection process for differentiated processing by subsequent modules.

[0037] This embodiment, through the above process, effectively filters noise and interference while ensuring the integrity of the original speech information, achieving high-quality standardized audio data input. This step not only improves the accuracy of subsequent speech recognition and semantic analysis but also lays a stable input foundation for emotion recognition and intent analysis. By combining segmentation markers with quality detection results, a traceable and evaluable input source can be provided for subsequent modules, thereby improving the reliability and adaptability of the overall processing flow.

[0038] S20, perform speech transcription operation based on the processed audio data to generate transcribed text, and extract acoustic features based on the processed audio data;

[0039] In this embodiment, the speech-to-text operation relies on processed audio data as input. This data has undergone noise reduction, segmentation, and quality checks, thus possessing high parsability. The first step in speech-to-text is loading a speech recognition model. This model can be an end-to-end recognition structure based on a deep neural network, supporting joint modeling of acoustic and linguistic features. During model loading, an initialization process is required, such as model parameter warm-up and cache establishment, to ensure stable response speed when processing continuous audio streams. Domain-specific terminology is introduced into the model, such as "deductible," "claim application," and "critical illness insurance." Hotword injection can improve the accuracy of recognizing insurance and medical terminology. During the transcription process, the generated text needs terminology correction, such as replacing misspellings caused by phoneme similarities and comparing and correcting fixed collocations defined in industry dictionaries, thereby obtaining accurate transcribed text.

[0040] Acoustic feature extraction is an operation performed simultaneously or in parallel with transcription. Common acoustic features include fundamental frequency parameters, energy parameters, and speech rate parameters. Fundamental frequency parameters primarily reflect the periodic characteristics of vocal tract vibrations and are extracted through autocorrelation functions or cepstral analysis. Energy parameters reflect sound intensity and are obtained through short-time energy or logarithmic energy calculations. Speech rate parameters are based on frame-level speech boundary detection to count the number of syllables or words per unit time. The extracted parameters often exhibit dimensional differences, thus requiring scale unification operations. For example, normalization, standardization, or min-max stretching methods can be used to unify the values ​​across different dimensions, allowing them to be processed within the same feature space. Finally, the unified fundamental frequency, energy, and speech rate features are fused to generate acoustic features, providing high-dimensional input for sentiment analysis and subsequent intent recognition.

[0041] Different speech recognition models can be selected for different scenarios. In environments with relatively clear speech, models combining convolutional neural networks and recurrent neural networks can be used to ensure low latency and high stability. In scenarios with high environmental noise, end-to-end speech recognition models based on the Transformer architecture can be adopted, using attention mechanisms to enhance the modeling ability of long-term dependencies. For industry terminology processing, a dictionary hot-update mechanism can be used to temporarily add newly emerging product or drug names during peak call center periods, thereby reducing the false recognition rate. In acoustic feature extraction, windowing and framing methods can improve the accuracy of fundamental frequency and energy calculations. Alternatively, a self-supervised deep learning model can be introduced to directly learn acoustic representations from audio waveforms, reducing the bias of manually defined features.

[0042] This embodiment significantly improves the accuracy of text transcription in financial and medical scenarios by introducing a domain-specific terminology vocabulary and performing terminology correction during the transcription process, avoiding the omission of key information. The combination of acoustic feature extraction and scale unification allows sentiment analysis to utilize both acoustic variations and semantic content simultaneously, thereby enhancing the ability to recognize complex emotional states. Speech transcription and acoustic feature extraction are performed in parallel, ensuring that the output contains both readable text and numerical features for sentiment modeling, guaranteeing the synergistic effect of semantic and sentiment analysis.

[0043] S30, perform sentiment analysis on the acoustic features and the transcribed text, and generate sentiment analysis results;

[0044] In this embodiment, sentiment analysis is based on the joint processing of acoustic features and transcribed text. Acoustic features include multi-dimensional parameters such as fundamental frequency, energy, and speech rate. These features reflect pronunciation intensity, intonation fluctuations, and language rhythm, and are important indicators for measuring emotional state. The content obtained from the transcribed text through speech recognition can carry semantic information; the word choices, sentence structures, and contextually implied semantic cues are often closely related to underlying emotional attitudes. The first step in sentiment analysis is to align the acoustic features in the temporal dimension, aligning the fundamental frequency curve, energy changes, and speech rate parameters with the timestamps of the transcribed text to generate aligned acoustic vectors, thereby ensuring that the acoustic signals and text semantics correspond within the same time frame.

[0045] Semantic analysis is then performed on the transcribed text to extract emotion-related words and contextual expressions. This process can be implemented using word vector models or pre-trained language models, ultimately generating a text sentiment vector. Acoustic vectors and text vectors belong to different modalities and require weighted fusion using an attention mechanism. This attention mechanism dynamically adjusts the weights of acoustic and textual information based on context; for example, when the tone is intense and contains complaining words, the fusion weight of the corresponding features is increased. The resulting multimodal feature matrix simultaneously preserves physiological signal changes in speech and semantic emotional cues in the text.

[0046] Composite emotion recognition is performed based on fused features, typically including determining the emotion category and quantifying the emotion intensity. Categories can be divided into different types such as anger, anxiety, and calmness, while intensity can be represented by a numerical range to indicate the degree of emotion's severity. Finally, the category and intensity are combined to form a sentiment analysis result that can be used for subsequent business logic.

[0047] Different emotion recognition models can be employed in different application environments. When audio data is short, a structure combining convolutional neural networks and recurrent neural networks can be used to quickly capture local changes and temporal dependencies in acoustic features. When speech data is long and emotional changes are subtle, a Transformer-based multimodal fusion model can be used to capture sentence-level emotional fluctuations through a self-attention mechanism. In the text analysis stage, a combination of an emotion dictionary and a deep learning model can be used to ensure high-precision recognition of fixed emotion words while also uncovering implicit emotional expressions through the model. Regarding the fusion method of acoustic and text features, direct concatenation, cross-attention, or gating fusion mechanisms can be selected, each corresponding to different computational complexity and recognition accuracy.

[0048] To adapt to various scenarios, parameter configurations also need to be adjusted. In the call center environment of the financial sector, where there is a lot of background noise, the robustness of acoustic features is particularly important, so the weights of energy and speech rate parameters can be increased. In the medical consultation environment, the patient's semantic expression is more critical, so the proportion of text features can be appropriately increased, and semantic continuity constraints can be added to the attention mechanism.

[0049] This embodiment, by jointly analyzing acoustic features and transcribed text, overcomes the shortcomings of single acoustic features, which are prone to misjudgment, and achieves more accurate identification of complex emotional states. The time alignment and fusion mechanism ensures that the speech signal and text semantics are processed within the same context, thereby improving the accuracy and stability of emotion recognition. Composite emotion recognition not only outputs the emotion category but also provides intensity information, offering a quantitative basis for subsequent business strategies such as urgency assessment.

[0050] S40, based on the transcribed text, perform intent and appeal recognition and extract the element set to generate intent and appeal recognition results;

[0051] In this embodiment, intent and request recognition relies on layer-by-layer processing of the transcribed text. First, intent classification is performed, mapping text fragments to predefined intent categories. Intent categories can cover different types such as reporting incidents, inquiries, complaints, claims, and progress inquiries. Intent classification is typically implemented using deep learning models, such as Transformer-based text classifiers, which can maintain recognition accuracy even with strong contextual dependencies. Before inputting the transcribed text, the classifier needs to segment and vectorize the text, then input it into the model to obtain the probability distribution of intent categories, thereby generating initial intent categories.

[0052] While identifying the intent category, it's necessary to identify key entities within the text. Named entity recognition (NER) is used to extract business category entities, numerical parameter entities, and event type entities. Business category entities can include insurance types, claims processes, and reimbursement recipients. Numerical parameter entities capture quantitative information such as amounts, times, and proportions. Event type entities reflect specific operations or contexts, such as hospitalization, car accidents, and refunds. NER can be implemented based on sequence labeling models, generating an initial set of elements by labeling the text character by character or word by word.

[0053] Because real-world user expressions often contain vague descriptions, it is necessary to further utilize a domain knowledge graph for parsing. Fuzzy expression parsing involves matching the initial intent category and initial element set with nodes in the knowledge graph to eliminate synonyms, colloquialisms, or abbreviations. For example, if a user says "reimbursement for medical expenses," the system will correct it to "medical expense reimbursement," thus obtaining the corrected intent category and corrected element set.

[0054] In multi-turn dialogue scenarios, the information extracted in a single turn is often incomplete, thus requiring multi-turn intent tracking based on a corrected set of elements. This process maintains the association between the set of elements in historical interactions and the current dialogue content, merging newly identified elements with historical content. For example, if the previous turn mentions "policy number" and the next turn mentions "claim amount," tracking can generate a cross-turn related set of elements.

[0055] Finally, the correction intent category is combined with the cross-round associated element set to generate structured intent and appeal recognition results, which serve as input for subsequent knowledge enhancement and tag generation.

[0056] Different implementation methods can be adopted in different application environments. Intent classification can be based on deep neural networks or a hybrid approach combining rules and models. For example, in insurance consultation, a rule base can prioritize recognizing fixed phrases such as "I want to report a crime," while a deep model can recognize complex contexts such as "What should I do if I'm in a car accident?" Named entity recognition can use conditional random field models or sequence labeling models fine-tuned from pre-trained language models. In fuzzy expression parsing, knowledge graph retrieval or graph neural networks can be used to improve the accuracy of fuzzy matching through graph structure learning.

[0057] In multi-round tracking, a dialogue state tracker can be used to store the elements identified in each round in a state store. Then, an attention mechanism is used to compare the current input with historical states, thereby achieving cross-round merging. For scenarios with high business adaptability requirements, rule weights can be introduced to prioritize the retention of the latest information for specific elements during merging.

[0058] In medical consultation scenarios, intent categories might include medical advice and feedback on drug side effects; numerical parameter entities need to focus on identifying elements such as dosage and duration. In financial claims scenarios, intent categories might include claims reimbursement and policy inquiries; numerical parameter entities need to identify elements such as claim amount and policy term. By adapting domain-specific knowledge graphs, the accuracy of fuzzy parsing and element tracking can be improved.

[0059] This embodiment extracts structured intent and appeal information from natural language by performing intent classification, named entity recognition, fuzzy parsing, and multi-round tracking on the transcribed text. Compared to single-round or single-recognition methods, this processing approach can dissolve ambiguity, reduce the omission of key information, and ensure the integrity of information across rounds, thereby providing high-quality input for subsequent knowledge-enhanced matching and label generation.

[0060] S50, perform enhanced matching between the sentiment analysis results and the intent and appeal recognition results in the knowledge base to obtain the knowledge base enhancement results;

[0061] In this embodiment, enhanced matching takes sentiment analysis results and intent and appeal recognition results as input, and completes further judgment and expansion through a knowledge base. First, business elements need to be scanned in the recognition results. Business elements typically include insurance type, reimbursement type, event time, amount parameters, etc. These elements are compared with a sensitive entity dictionary to identify entities that belong to personal identity information, such as names, ID numbers, and contact information, and generate a sensitive entity tag set for them. The generation of the sensitive entity tag set is not only a requirement for data protection, but also provides a foundation for subsequent risk control.

[0062] After sensitive entity identification is completed, the business knowledge graph in the knowledge base is invoked to perform business strategy mapping. The business knowledge graph models entities and relationships, organizing information such as claims processes, reporting procedures, and insurance terms into a graph structure. The mapping operation matches the identified business elements with nodes in the knowledge graph to obtain the business strategy mapping results. For example, a mapping relationship is established between "hospitalization reimbursement" and "medical insurance terms," ​​ensuring that the identification results correspond to actual business rules.

[0063] The sentiment analysis results are then fed into the compliance module, where sentiment compliance checks are performed against the regulatory terms library. This library contains provisions regarding service quality, complaint handling, and customer reassurance. When an emotional state conflicts with these terms, the system generates a sentiment compliance flag. For example, in insurance services, if anger is detected as the emotional state, and the terms require immediate transfer to a human agent in angry complaint scenarios, the system will add a compliance warning to the result.

[0064] After obtaining the sensitive entity tag set, business strategy mapping results, and sentiment compliance tags, anomaly level analysis is further performed. This analysis combines sensitivity, business mapping results, and sentiment compliance status to calculate whether the current interaction presents a high risk or emergency, generating anomaly level analysis results. The analysis results can be expressed as a quantitative value representing risk intensity or as a graded representation of anomaly level.

[0065] Finally, the sensitive entity tag set, business strategy mapping results, sentiment compliance tags, and anomaly level analysis results are merged to generate a unified knowledge base enhancement result. This fusion is not merely a simple merger, but rather an integration of information from different sources through weighted or rule-based methods to ensure that the enhancement result comprehensively reflects customer intent, sentiment, and business risks.

[0066] Different implementation methods can be adopted in different environments. Sensitive entity identification can be achieved through dictionary matching based on regular expressions, or through deep sequence labeling models to identify potentially sensitive information. In business strategy mapping, a retrieval mechanism based on graph databases can be used, or graph neural networks can be used to enhance matching capabilities in complex scenarios. Sentiment compliance verification can achieve rapid detection based on rule matching, or a compliance classification model can be introduced to automatically learn the correspondence between different emotional states and clauses. Anomaly level analysis can be performed using a rule-weighted approach, simply adding together the sensitivity and emotional intensity, or a multi-dimensional risk scoring model can be used to generate a more accurate anomaly level.

[0067] In the claims service scenario within the financial business, knowledge base enhancements can be used to merge the identification results of "high claim amount + anger + involving key customers" into high-risk events, prompting rapid intervention. In service consultations within the healthcare field, "feedback on drug side effects + anxiety + involving sensitive medical records" can be marked as abnormal situations requiring focused recording and feedback.

[0068] This embodiment performs enhanced matching in the knowledge base, which supplements and corrects the sentiment and intent recognition results, achieving comprehensive coverage of sensitivity, compliance, and business mapping. Compared to directly outputting intent and sentiment results, the results generated by enhanced matching are more structured and reliable, can resolve ambiguity in user expression, and quantify risk levels, providing solid data support for subsequent tag generation.

[0069] S60, based on the knowledge base enhancement results, generate a set of tags including call type, core demand, emotional state, urgency level, and sensitivity level.

[0070] In this embodiment, the process of generating the tag set takes the knowledge base enhancement results as input. By parsing and mapping the structured information contained therein, multi-dimensional tags such as emotional state, urgency level, and sensitivity level are output. First, in the emotional state dimension, the emotional compliance markers in the enhancement results need to be integrated with the emotional analysis results, and emotional state tags are generated through feature matching and rule mapping. Emotional state tags not only represent basic emotions such as anger, anxiety, and calmness, but can also be further subdivided according to emotional intensity and contextual cues, such as "mild dissatisfaction" or "intense anxiety".

[0071] The generation of urgency level labels relies on the results of anomaly level analysis, combined with business strategy mapping results and sentiment status labels. Boundaries are set using dynamic thresholds; for example, a risk value exceeding a certain range is labeled as high urgency, while a value in the middle range is labeled as medium urgency. The dynamic thresholds can be adjusted using historical interaction data and business feedback to refine the grading boundaries, ensuring that the urgency level classification adapts to changes in different time periods and business scenarios.

[0072] The generation of sensitivity level labels primarily relies on the sensitive entity tag set and regulatory requirements. By comparing and enhancing the scope and number of sensitive entities involved in the results, and combining compliance rules, a level such as core, important, sensitive, or general is output. For example, interactions involving customer policy numbers and medical diagnosis results may be labeled as core, while interactions containing only general business parameters may be labeled as general. The classification of sensitivity levels needs to balance compliance risks and information security requirements, avoiding over-labeling or omission.

[0073] Finally, the emotional state tags, urgency tags, and sensitivity level tags are combined to form a tag set. This set is output in a structured form and can be directly accessed by downstream customer service dispatch, quality inspection systems, or risk warning platforms to support business implementation.

[0074] Different implementation methods can be adopted in different scenarios. The generation of emotion state labels can be achieved through a multimodal fusion classification model, using acoustic and textual features as input for joint judgment. Alternatively, a rule-matching model can be used for rapid application in small-scale scenarios. The classification of urgency labels can be based on a score threshold method, or a deep learning model can be used to regress and predict risk levels, then the predicted values ​​can be mapped to graded labels. Sensitivity level labels can be implemented through a manually defined rule system, such as keyword matching and grading, or a machine learning model combined with a knowledge graph can be used to identify sensitive information in complex scenarios.

[0075] In financial scenarios, claims calls may include monetary parameters, claims progress, and emotional expressions. By enhancing result analysis, the system can generate emotional state labels such as "anxiety," urgency labels such as "high," and sensitivity labels such as "critical," thereby accelerating claims processing. In healthcare scenarios, when patients inquire about drug reimbursement and adverse reactions, the system may generate emotional state labels such as "worry," urgency labels such as "moderate," and sensitivity labels such as "important," prompting medical support personnel to prioritize answering and recording these inquiries.

[0076] This embodiment generates a multi-dimensional tag set based on knowledge base enhancement results, enabling the expression of emotional state, urgency, and sensitivity level in a structured form. This allows interactive content to move beyond mere text understanding and generate tag outputs that directly drive business processes. This approach eliminates the limitations of single emotion recognition or single intent recognition, allowing the system to establish an effective correlation between compliance, urgency, and emotion perception, thereby improving the accuracy and timeliness of risk warnings, claims processing, and customer service.

[0077] This invention relates to the field of artificial intelligence technology and can be applied to business scenarios such as fintech and healthcare. It discloses a method, apparatus, device, and medium for generating tags based on emotion recognition, comprising: acquiring target audio and performing audio preprocessing to obtain processed audio data; performing speech-to-text transcription based on the processed audio data to generate transcribed text and extracting acoustic features; performing emotion analysis on the acoustic features and transcribed text to generate emotion analysis results; performing intent and appeal recognition based on the transcribed text and extracting a set of elements to generate intent and appeal recognition results; performing enhanced matching on the emotion analysis results and intent and appeal recognition results within a knowledge base to obtain knowledge base enhanced results; and generating a tag set based on the knowledge base enhanced results. This invention achieves a synergistic improvement in emotion analysis and intent recognition by jointly processing the acoustic features of audio data and the transcribed text, and generates multi-dimensional tags by combining knowledge base enhanced matching. This not only improves the accuracy and industry adaptability of audio recognition during the audio understanding process but also forms structured tag outputs that can be directly used in business applications, enhancing the value conversion capability of audio data.

[0078] In one embodiment, step S10 above includes:

[0079] S101, Obtain the target audio;

[0080] S102, Perform noise suppression operation on the target audio to generate noise-reduced audio data;

[0081] S103, Perform speaker separation operation on the noise-reduced audio data to generate separated audio segments;

[0082] S104, Perform speech segmentation on the separated audio segments to generate segmented audio data;

[0083] S105, perform an audio quality detection operation on the segmented audio data, add segmentation marks to the segmented audio data that passes the audio quality detection, and generate processed audio data.

[0084] In this embodiment, the target audio processing objects cover three types: local recording, call recording, and streaming input. A unified access interface standardizes the sampling rate, bit depth, and number of channels, and simultaneously generates acquisition metadata for subsequent tracing and alignment. To reduce time-domain and frequency-domain deviations in subsequent processing, a time baseline is established during the access phase, mapping the input timestamps to a unified time axis to form an indexable original trajectory. Here, the target audio refers to both complete conversations and passively triggered fragmented recordings, which can be uploaded by agent terminals or pushed by business systems. After standardization, it enters the enhanced link.

[0085] The purpose of noise suppression on the target audio is to suppress steady-state and non-steady-state interference while preserving speech details. The implementation path employs a dual-channel design combining time-frequency masking and time-domain enhancement. First, a speech mask is estimated in the frequency domain to reduce environmental noise and echo remnants. Then, waveform-level correction is performed in the time domain to restore formants and transient structures. Masking estimation incorporates speech and noise priors, utilizing joint constraints of amplitude spectrum and phase difference to improve suppression selectivity. Loudness preservation and distortion constraints are added during waveform correction to avoid over-smoothing of high-energy consonants. The output noise-reduced audio data retains the entire timescale and channel mapping information, which is directly reused in subsequent links for segment localization.

[0086] The goal of speaker separation on noise-reduced audio data is to distinguish different speech sources and form traceable speaker identifiers. In single-channel scenarios, speaker embedding vectors are first extracted, then speaker clusters are obtained based on similarity clustering, and sequence constraints are used for redistribution to reduce short-term jitter. In dual-channel scenarios, physical channels are used first for initial separation, and then embedding consistency checks are introduced to correct for cross-speech omissions. To alleviate cross-segment jumps by the same speaker, intra-segment and inter-segment coherence measures are added, and the attribution near the boundary is re-estimated. The output separated audio segments carry speaker identifiers, start and end times, and source trajectories, ensuring that subsequent segmentation and quality inspection can be converged based on a unified identifier.

[0087] The system performs speech segmentation on separated audio segments, focusing on semantic unit segmentation and silence boundary localization. The processing flow incorporates joint determination of endpoint detection and semantic pause detection. First, coarse localization is performed based on energy, spectral tilt, and zero-crossing rate. Then, word-level rhythmic cues and short pause patterns are used for further subdivision to match natural speech flow. To avoid recognition instability caused by excessively short segments, dual boundaries of minimum duration and minimum interval are set. Adjacent units are merged preferentially when encountering consecutive weak pauses. To suppress emotional drift caused by excessively long segments, secondary boundaries are added within long segments. The output segmented audio data retains the segment order and parent-child relationships, facilitating subsequent dialogue reconstruction.

[0088] Audio quality inspection is performed on segmented audio data to remove unusable segments and identify areas for repair. The quality inspection uses a set of indicators across three dimensions: intelligibility, completeness, and stability. Intelligibility assesses speech presence and language separability; completeness assesses clipping, packet loss, and discontinuity; and stability assesses background noise reverberation and bandwidth anomalies. Each indicator employs a joint decision-making process using decision boundaries and confidence levels, generating two results: pass and fail. Boundary samples are marked as review candidates for secondary inspection. Segmented audio data that passes the quality inspection is then tagged with segment labels containing segment number, speaker identifier, start and end times, source trajectory, processing chain summary, and confidence level, establishing a traceable index between segments and the processing process. Segment tags adhere to the principles of unambiguity and composability, ensuring that the same segment can be uniquely referenced in any subsequent stage and aligned with upstream and downstream results.

[0089] The final processed audio data consists of segmented audio data that has passed quality checks and corresponding segment tags. It also preserves the reference relationships between the noise-reduced audio data and the separated audio segments, forming a directed mapping from the original trajectory to usable segments. This data format supports parallel access for subsequent speech transcription and acoustic feature extraction, supports dialogue reconstruction and playback, and supports aggregated statistics based on speaker identifiers. It ensures that both text and acoustic processes can operate under a unified time scale and avoids redundant preprocessing.

[0090] This embodiment reduces interference and preserves speech details through noise suppression, improving the intelligibility and stability of speech transcription and acoustic feature extraction. Speaker separation enables source differentiation, allowing intent extraction and emotion discrimination to aggregate at the correct speaker dimension, reducing cross-speaker contamination. Speech segmentation obtains semantically and rhythmically appropriate units, enhancing the consistency of text alignment and acoustic statistics boundaries, reducing the dual risks of long-segment drift and short-segment instability. By combining audio quality detection with segmentation labeling, a one-to-one correspondence is established between available segments and processing links, allowing subsequent analysis modules to directly reuse segmentation labels for alignment and recall, avoiding duplicate judgments and misuse of low-quality data.

[0091] In one embodiment, step S20 above includes:

[0092] S201, Load the customized speech recognition model, perform a preheating initialization operation on the customized speech recognition model, and inject an industry terminology vocabulary into the customized speech recognition model;

[0093] S202, the customized speech recognition model is used to perform speech transcription on the processed audio data, and the speech transcription result is subjected to terminology correction processing to generate transcribed text;

[0094] S203, extract fundamental frequency feature parameters, energy feature parameters, and speech rate feature parameters from the processed audio data;

[0095] S204, perform a scale unification operation on the fundamental frequency feature parameters, energy feature parameters and speech rate feature parameters, and fuse the scale-unified fundamental frequency feature parameters, energy feature parameters and speech rate feature parameters to generate acoustic features.

[0096] In this embodiment, a customized speech recognition model is loaded to perform parameter and vocabulary adaptation for the industry discourse system. The model file, decoding configuration, and hot word container are loaded into the inference environment together, completing dependency checks and resource binding between the acoustic front-end, language decoder, and post-processing module. During the loading phase, a unified timeline and segmentation marker mapping relationship is established to ensure that subsequent text and acoustic measurements are aligned on the same timeline. The inference environment completes device detection and computing power allocation, enabling parallel streams as needed to support multi-segment concurrent processing.

[0097] A warm-up initialization process is performed on the customized speech recognition model to eliminate cold-start jitter and initial decoding lag. This warm-up process uses short silence segments and a small number of effective speech segments to drive the construction of a full-chain cache for front-end feature extraction, acoustic coding, decoding search, and text post-processing. The cache includes the working states of convolution and attention, commonly used paths in the language search graph, word-level regularization tables, and the word segmenter context window. After warm-up, the model maintains the context window within the same session to accommodate cross-segment contexts.

[0098] Industry-specific terminology vocabularies are injected into a customized speech recognition model to improve the recall and ranking of specialized vocabulary. The vocabulary consists of three types of entries: lexical sequences, word boundaries, and pronunciation maps. During injection, the state transition probabilities of lexical units and the language decoder are weighted, and distinction constraints are set for words with similar pronunciations and forms to reduce false triggers. A reference relationship is established between the vocabulary and segment tags to ensure that different word sets can be selectively used for different sessions and different business lines. After injection, the terms take effect simultaneously in both the decoding search and candidate re-ranking stages.

[0099] When performing speech-to-text transcription on processed audio data using a customized speech recognition model, segmented audio data is first read based on a unified timeline. The acoustic front end extracts time-frequency features from the waveform and performs channel normalization. The acoustic encoder outputs frame-level representations, and the language decoder generates candidate sequences within the search space and outputs word-level alignments with confidence scores. To reduce drift introduced by accents and pauses, a boundary stabilization strategy is implemented for long pauses, and an adaptive decoding step size strategy is implemented for fast speech rates. The transcription output carries a timestamp, speaker identifier, and segment number for subsequent text alignment and sentiment fusion.

[0100] The speech-to-text transcription results undergo terminology correction to rectify near-sound substitution and word segmentation errors. The correction process first performs substring matching and pronunciation comparison between the candidate sequence and the industry terminology vocabulary. Then, low-confidence segments are rearranged, prioritizing terminology segments consistent with the business context. The corrected text is then used to generate the transcribed text, retaining the correction trajectory and confidence changes for subsequent quality traceability and manual sampling.

[0101] Fundamental frequency (PF) feature parameters, energy feature parameters, and speech rate feature parameters are extracted from the processed audio data to form text-independent acoustic cues. PF feature parameters are obtained through joint estimation of autocorrelation and cepstral signatures to obtain the pitch period and silent frame markers, supplemented by an octave suppression strategy to avoid abrupt transitions. Energy feature parameters measure speech intensity and resonance distribution based on the ratio of short-time energy to band energy, and smooth the energy envelope within segments to preserve transient changes. Speech rate feature parameters estimate the number of word boundaries per unit time through rhythmic peaks and boundary detection, and output a pause distribution histogram to characterize fluency and tension. All three types of parameters are bound to segment markers and a unified timeline to ensure consistency with text alignment.

[0102] To eliminate dimensional and individual differences, a scaling operation was performed on the fundamental frequency, energy, and speech rate characteristics. First, duration and amplitude were normalized for each segment. Then, a reference baseline was constructed at the conversation level. A strategy combining parameter-free interval stretching and robust standardization was used to output the scale-normalized feature sequence. The three types of scale-normalized sequences were aligned frame-by-frame on the time axis, balancing intra-segment variations and inter-segment continuity. Subsequently, the scale-normalized fundamental frequency, energy, and speech rate characteristics were fused. The fusion process involved frame-by-frame splicing and incorporating in-window statistics and rates of change to generate acoustic features for subsequent analysis. These acoustic features carried time indices and segment numbers, facilitating bidirectional retrieval and alignment with the word timestamps of the transcribed text.

[0103] This embodiment maintains stable output in terms of latency and professional vocabulary recall through preheating initialization and terminology vocabulary injection, reducing cold start jitter and near-sound replacement. Through terminology correction processing, low-confidence segments in the candidate sequence are reordered in a targeted manner, making the transcribed text more reliable in the industry context. By extracting and unifying the scale of fundamental frequency feature parameters, energy feature parameters, and speech rate feature parameters, acoustic features are comparable in terms of dimensionality and individual differences, making subsequent alignment with text timestamps more accurate. By forming parallel and searchable representation pairs between the fused acoustic features and the transcribed text, subsequent sentiment analysis and intent recognition can be directly referenced bimodally based on a unified timeline, reducing the cost of repeated preprocessing and cross-module alignment, and improving overall processing throughput and stability.

[0104] In one embodiment, step S30 above includes:

[0105] S301, Perform feature alignment processing on the acoustic features to generate aligned acoustic feature vectors;

[0106] S302, Perform sentiment semantic analysis on the transcribed text to generate a text sentiment feature vector;

[0107] S303, assign feature weights to the aligned acoustic feature vector and the text sentiment feature vector through an attention mechanism to generate a weighted acoustic feature vector and a weighted text feature vector;

[0108] S304, the weighted acoustic feature vector and the weighted text feature vector are fused to generate a fused feature matrix;

[0109] S305, Perform a composite emotion recognition operation on the fused feature matrix to obtain the basic emotion type and emotion intensity;

[0110] S306, combine the basic emotion type and emotion intensity to generate emotion analysis results.

[0111] In this embodiment, the sentiment analysis stage uses acoustic features and transcribed text as parallel inputs, first establishing a unified temporal and semantic baseline. Acoustic features are derived from frame-level sequences such as fundamental frequency, energy, and speech rate, carrying segment numbers and time indices; transcribed text is derived from speech transcription output, carrying word-level timestamps and speaker markers. To avoid misalignment in sampling intervals and semantic granularity between the two types of inputs, feature alignment is first performed on the acoustic features: using the word boundaries of the transcribed text as anchor points, adjacent acoustic frames are aggregated into corresponding word intervals according to their timestamps, and statistical quantities such as mean, variance, peak value, and rate of change, as well as short-window dynamics, are generated for each interval, preserving continuous changes across words. Silence and padding masks are introduced during alignment to ensure that speech-free intervals do not interfere with subsequent representations. After aggregation and normalization, aligned acoustic feature vectors are formed, with fixed dimensions and an order strictly corresponding to the word or sub-word order of the text sequence.

[0112] Sentiment semantic analysis is performed on the transcribed text. First, the text is standardized (unified number pronunciation, normalized colloquial expressions, and folded repeated words). Then, it is segmented into words or sub-words to generate a sequence of semantic fragments. Each semantic fragment is embedded and mapped to form a context-dependent representation, maintaining a one-to-one correspondence with word timestamps and preserving sequence information through positional encoding. Based on the context window, the relationships between sentiment indicators such as negation, transition, and mitigation are modeled to generate a text sentiment feature vector. This vector is aligned with the aligned acoustic feature vector in length and index, facilitating subsequent position-by-position fusion.

[0113] To highlight key cues and suppress redundant signals, an attention mechanism is introduced to assign feature weights to two types of vectors. Using the text sentiment feature vector as the query channel and the aligned acoustic feature vector as the attention channel, the relevance weights between positions are learned, allowing semantically significant segments (e.g., those containing strongly directional emotional words or complaint cues) to reinforce acoustic cues (such as sudden energy spikes or rhythmic abrupt changes) within their temporal neighborhood. Simultaneously, symmetrical weighting is applied in the acoustic-to-text direction, enabling significant acoustic anomalies to increase the importance of corresponding text segments. After bidirectional weighting, weighted acoustic feature vectors and weighted text feature vectors are obtained; both retain their original temporal indices and masks to ensure that invalid regions are not included in subsequent calculations.

[0114] In the fusion stage, the weighted acoustic feature vector and the weighted text feature vector are aligned along the positional dimension, then concatenated and linearly mapped. The aggregated values ​​from the local context window and the global summary across windows are then superimposed to form a fused feature matrix. The rows of this matrix follow the temporal order of the text, and the columns cover the acoustic, textual, and cross-modal interaction dimensions. Segment boundaries and speaker switching markers are preserved to facilitate subsequent segment-level and session-level discrimination.

[0115] A composite emotion recognition operation is performed on the fused feature matrix, outputting two quantities: basic emotion type and emotion intensity. The basic emotion type first generates position-by-position candidate assignments at the location dimension, then aggregates into stable labels at the segment dimension, handling the coexistence and transition of multiple emotions within the same time period. Emotion intensity is represented by a normalized interval to indicate relative strength, while also providing peak position and duration information to reflect the trajectory of emotion fluctuations. Smoothing and consistency constraints are applied to continuous speech segments to avoid label jumps caused by momentary jitter. Finally, the basic emotion type and emotion intensity are combined in chronological order to generate the emotion analysis result; this result includes segment-level emotion labels, intensity curves, time indexes, and speaker-related entries, which can be directly retrieved and referenced by downstream intent and label generation processes.

[0116] This embodiment achieves consistent expression of acoustic and semantic cues by aligning acoustic feature vectors and textual sentiment feature vectors through bidirectional weighting and temporal alignment. The fused feature matrix simultaneously covers tone variations and semantic tendencies. The composite discrimination outputs both the basic emotion type and emotion intensity, avoiding biases caused by using only a single acoustic cue and the inadequacy of ignoring tone strength when using only textual cues. This maintains label stability in environments with long conversations, multiple speakers, and uneven speech rates, reducing misjudgments and omissions, and providing temporally accurate and granular emotion input for subsequent intent recognition and label generation.

[0117] In one embodiment, step S40 above includes:

[0118] S401, Perform an intent classification operation on the transcribed text to generate an initial intent category;

[0119] S402, Perform named entity recognition operation on the transcribed text to extract business category entities, numerical parameter entities and event type entities, and generate an initial element set;

[0120] S403, perform fuzzy representation parsing operation on the initial intent category and the initial element set based on the domain knowledge graph to generate the corrected intent category and the corrected element set respectively;

[0121] S404, Perform multi-round intent tracking operation on the set of correction elements, associate historical interaction element relationships, and generate a cross-round associated element set;

[0122] S405, combine the corrected intent category and the cross-round associated element set to generate intent and appeal recognition results.

[0123] In this embodiment, intent and appeal recognition uses transcribed text as the sole input. First, a unified view of the text expression is constructed, unifying sentence segmentation, paragraphing, speaker tagging, and time indexing. Duplicate blanks and colloquial filler words are removed, while business-related quantitative phrases, event phrases, and category phrases are retained. Based on this view, intent classification is performed, establishing a dialogue turn-oriented discrimination mechanism: semantic representation and contextual representation are generated for each semantic segment, and segment-level discrimination is formed by combining speaker switching and the positional relationship of semantic connectors. Subsequently, consistency constraints and conflict resolution are performed at the turn level, outputting an initial intent category. This category is also accompanied by confidence level, time range, and triggering segment index for subsequent alignment with the element set.

[0124] Named entity recognition (NID) revolves around three dimensions: business category entities, numerical parameter entities, and event type entities. Business category entities indicate the business branch and processing scenario, numerical parameter entities carry quantifiable information such as amount, quantity, limit, and deadline, and event type entities carry event references such as accidents, malfunctions, complaints, and reimbursements. To ensure the composability of subsequent intent classification results, entity extraction is performed on the same semantic segmentation and speaker tagging, outputting an initial set of elements. The set elements include entity text, standardized semantic tags, occurrence location, and contextual trigger words. Ambiguous quantified phrases are appended with unit normalization and interval merging tags to ensure the comparability of numerical expressions across rounds of association.

[0125] Domain knowledge graphs are used to handle fuzzy expressions and cross-concept mappings. Initial intent categories and initial element sets are mapped to concepts, attributes, and relationships in the graph, with semantic convergence and alias alignment based on concept adjacency and relationship strength. Cross-segment referential bindings are introduced for expressions containing pronouns, ellipsis, and implicit references, allowing objects not explicitly appearing to be located to existing concept nodes via relational chains. This process generates two types of outputs: corrected intent categories and corrected element sets. The former refines or merges existing categories, while the latter aligns business category entities, numerical parameter entities, and event type entities with standard names, unifies units, and completes relationships. If graph relationships conflict with contextual triggering segments, they are resolved based on dialogue temporal proximity and speaker continuity, ensuring the output can be used for subsequent rounds of tracking.

[0126] Multi-turn intent tracking unfolds across conversation history, matching the set of corrected elements with the relationships between historical interaction elements. It utilizes the dialogue timeline, speaker alternation, and sequential dependencies to construct cross-turn element pointers. Similar entities are merged, and equivalent links are established for different expressions of the same entity. Drift of numerical parameters within the timeline is monitored, and the latest valid values ​​and change trajectories are retained. To handle breaks caused by jumps and interruptions, context windows and cross-window bridging are introduced to ensure that distant references fall back into the same conceptual chain. After tracking is complete, a cross-turn related element set is formed. Each element in the set records its source turn, latest standard name, valid time range, and dependency on the intent category, providing traceable data for the final output.

[0127] The final stage combines the corrected intent category with the cross-round associated element set to generate intent and appeal identification results. Two constraints are followed during the combination: first, the consistency constraint between category and element, rejecting entity pairings that contradict the category definition; second, the consistency constraint between time and speaker, not merging mutually exclusive time periods or sources. A resolution order is provided for residual conflicts: first, numerical parameter inconsistencies are addressed, then event type inconsistencies, and finally, inconsistencies in business categories. The output includes category labels, an element list, a time index, speaker mappings, and a confidence matrix, maintaining consistency with the fragment-level index of the transcribed text for easy alignment and joint use with sentiment analysis results in subsequent stages.

[0128] This embodiment performs intent classification and entity extraction under a unified view, uses a domain knowledge graph for fuzzy representation parsing and alias alignment, and combines multi-turn tracking to generate cross-turn related element sets. This enables the aggregation of business clues scattered across different segments, turns, and speakers into a clearly structured, time-traceable, and semantically consistent recognition result. Compared to directly judging based on single-turn text, this process reduces errors caused by category drift and missing referents, lowers the probability of element omissions due to omissions and pronouns, and ensures that intent categories and element sets remain stable and consistent in long dialogues and interrupted scenarios, providing more accurate and complete input for knowledge base enhancement and tag generation.

[0129] In one embodiment, step S50 above includes:

[0130] S501, Scan the business elements in the intent and request identification results, identify the personal identity information entities in the business elements through the sensitive entity dictionary, and generate a sensitive entity tag set;

[0131] S502, Perform a business strategy mapping operation on the intent and request identification results based on the domain knowledge graph to generate a business strategy mapping result;

[0132] S503, Perform an emotion compliance verification operation on the emotion analysis results according to the regulatory clause library, detect conflicts between the emotional state and regulatory requirements, and generate an emotion compliance marker;

[0133] S504, Based on the sensitive entity tag set, business strategy mapping results and sentiment compliance tags, perform an anomaly level analysis operation to generate an anomaly level analysis result;

[0134] S505, integrate the sensitive entity tag set, business strategy mapping results, sentiment compliance tags, and anomaly level analysis results to generate knowledge base enhancement results.

[0135] In this embodiment, enhanced matching uses sentiment analysis results and intent / appeal recognition results as parallel inputs, and alignment is first performed in a unified data view. The unified data view aligns the timestamps, speaker tags, and fragment indices of the two inputs, establishes a mapping table from fragments to timelines, and maps category labels, element lists, sentiment types, and sentiment intensities to the same timeline and the same session identifier, forming a searchable key-value structure for easy subsequent querying and cross-referencing. Subsequently, a scan is performed on the business elements in the intent / appeal recognition results. Business elements include business category entities, numerical parameter entities, and event type entities. The scanning process traverses the element list at the fragment granularity, extracts the original text, standard names, units, and contextual trigger words, constructs the sequence to be detected, and records the time range, speaker, and confidence level of each element.

[0136] Sensitive entity identification is performed based on a sensitive entity dictionary, which contains standard names, aliases, mask templates, and cross-domain equivalence terms for personally identifiable information entities, such as ID numbers, contact information, account identifiers, and medical record numbers. The sequence to be detected is matched against dictionary entries and templates item by item, employing a combination of exact matching and pattern matching to identify potential entities. Relaxed matching is introduced for morphological variations and delimiter insertions, and common character substitutions introduced by transcription are equivalently merged. The identification results are output as a set of sensitive entity tags. Each tag includes entity type, hit segment, hit location, standard name, mask location information, and hit confidence, and is associated with the segment index in a unified data view to ensure traceability for subsequent fusion and tracking.

[0137] Business strategy mapping is performed based on a domain knowledge graph. The domain knowledge graph organizes industry-standard business concepts, attributes, and relationships, including connections between business categories and processing flows, connections between event types and triggering conditions, and subordinate relationships between numerical parameters and threshold ranges. After mapping the categories and elements from the intent and request identification results to graph nodes, reasoning is performed along the relationship edges to obtain process nodes, required materials, priority clues, and cross-departmental collaboration entry points related to the current element combination. For categories and elements with aliases or abbreviated expressions, alias normalization and abbreviation completion are performed first, followed by relationship queries to ensure that the mapping results can be used for subsequent hierarchical and fusion processes. The output is represented as the business strategy mapping result, which includes the hit process nodes, triggering conditions, material list placeholders, and suggested priority clues, and records the source element and relationship path length for subsequent weight allocation.

[0138] Emotional compliance verification is conducted based on a regulatory clause database. This database stores clause entries, applicable scenarios, restrictions, and sensitive emotional trigger points. Emotion types and intensities are matched against clause entries, and flags are output for situations where emotional intensity exceeds a threshold or where the combination of emotion type and business scenario is unsuitable for continued automatic processing. During verification, the speaker's identity and temporal context are also considered. A cumulative count is built for consecutive high-intensity negative emotions within the same conversation, and an escalation flag is output when the cumulative threshold is reached. Verification results are represented by emotional compliance flags, including the matched clause entry, the trigger time range, the cumulative count, and the suggested handling level.

[0139] Anomaly level analysis takes a set of sensitive entity tags, business strategy mapping results, and sentiment compliance tags as input, and outputs anomaly level analysis results. The analysis process constructs three components: a sensitive component, a process component, and a sentiment component. The sensitive component estimates exposure intensity based on sensitive entity type and mask coverage; the process component estimates handling risk and time pressure based on process nodes and triggering conditions in the business strategy mapping results; and the sentiment component estimates dialogue risk based on sentiment type, sentiment intensity, and their cumulative count. The three components are normalized under a unified scale and then weighted and synthesized. The weights can be adjusted by the relational path length and confidence provided by the domain knowledge graph. Temporal aggregation is performed on multiple tags that overlap in time to ensure that the anomaly level has a unique and stable output within the same time window. The anomaly level analysis results include a window-level level value, the composition of contributing components, and an index of suggested handling strategies.

[0140] The integration phase writes the sensitive entity tag set, business strategy mapping results, sentiment compliance tags, and anomaly level analysis results into a unified data view, performing cross-source consistency checks and conflict resolution. Priority is given to ensuring consistency in time, speaker, and segment source. When conflicts occur, windows with higher anomaly levels take precedence, mappings with stricter process components take precedence, and masking of lower-sensitivity components takes precedence, generating knowledge base enhancement results. These enhanced results are output as structured objects, including sensitive annotations for the business element list, process suggestions, compliance module outputs, and anomaly level timelines, along with cross-source references and a confidence matrix for direct reference in subsequent tag generation.

[0141] This embodiment aligns sentiment analysis results with intent and appeal identification results in a unified data view. It introduces a sensitive entity dictionary, domain knowledge graph, and regulatory clause library to complete personal information identification, business strategy mapping, and sentiment compliance verification, respectively. Then, it synthesizes anomaly levels using a fractional approach and performs consistent fusion. This transforms scattered semantic, process, and sentiment cues into structured, traceable, and prioritized enhanced results. This allows for direct determination of label dimensions and grading criteria in subsequent stages, reducing misjudgments caused by terminological ambiguity, unclear processes, and accumulated sentiment. It also improves the ability to capture high-risk windows and highly sensitive elements, and provides clear and actionable input for both manual intervention and automated processing.

[0142] In one embodiment, step S60 above includes:

[0143] S601, Perform a call type mapping operation on the knowledge base enhancement result to generate call type labels;

[0144] S602, Perform a business request parsing operation on the knowledge base enhancement results to generate core request tags;

[0145] S603, Perform an emotion state recognition operation on the knowledge base enhancement result to generate emotion state labels;

[0146] S604, Perform an urgency level classification operation on the knowledge base enhancement results based on a dynamic threshold to generate urgency level labels;

[0147] S605, Perform a sensitivity level classification operation on the knowledge base enhancement results based on the compliance strategy to generate sensitivity level labels;

[0148] S606, Perform tag association analysis on the call type tag, core demand tag, emotional state tag, urgency tag and sensitivity level tag, and establish a tag logical relationship strategy;

[0149] S607, Perform a tag conflict verification operation based on the tag logical relationship strategy to generate a conflict verification result;

[0150] S608, adjust the call type label, core demand label, emotional state label, urgency label and sensitivity level label based on the conflict verification result to generate a structured label set.

[0151] In this embodiment, the input is the knowledge base enhancement result, which includes sensitive annotations of the business element list, process suggestions, compliance verification outputs, and anomaly level timelines, and is displayed in the unified data view with timestamps, speaker tags, and fragment indexes. The call type mapping operation uses the unified data view as the retrieval entry point, establishing a bidirectional index between semantic fragments and a mapping terminology list. The mapping terminology list is derived from question-and-answer corpora and process terminology lists from business scenarios such as insurance, healthcare, and financial services, covering equivalent and abbreviated expressions for reporting, consulting, and complaining. Semantic similarity and a trigger table are calculated for each fragment. The trigger table records the simultaneous occurrence relationship between trigger words, modal particles, and context windows, outputting call type labels and retaining the trigger basis and confidence level. The business request parsing operation extracts target objects and action phrases from the process suggestions and element list, pairs them using predicate-argument structures and knowledge graph relationship edges, combines elements into executable request categories, outputs core request labels, and records the pointing relationship with process nodes and material list placeholders. The emotion state recognition operation directly reuses the type and intensity trajectories from the emotion analysis results, performs time alignment and speaker filtering in a unified data view, and generates emotion state labels using window aggregation and peak preservation strategies, while retaining the time interval of peak occurrence and the index of the affected dialogue segments.

[0152] The urgency grading process is driven by dynamic thresholds. These thresholds are derived from cluster centers of anomaly level timelines, processing timeliness clues in workflow suggestions, and historical handling records. Grading boundaries are generated through multi-source indicator normalization and weighted synthesis. At each time window, anomaly level and workflow time limit signals are synthesized to generate an urgency label, along with a grading boundary version number and decision path, ensuring traceability and continuous updates. Sensitivity level classification is based on compliance rules derived from regulatory clauses, sensitive entity dictionary classifications, and cross-domain equivalence mapping. The sensitive entity tag set is combined and matched with the business context to determine if a higher protection level is reached. The result is written into a sensitivity level label, along with masking suggestions and prompts for implementing the principle of minimal disclosure.

[0153] The tag association analysis operation constructs a relationship graph among five basic tags. The graph nodes represent call type, core demand, emotional state, urgency, and sensitivity levels, while the edges represent statistical co-occurrence, process dependency, and compliance constraints. Statistical co-occurrence is derived from historical annotations and online accumulation of occurrence relationships; process dependency is derived from the sequence and necessary conditions of the knowledge graph; and compliance constraints are derived from clauses and internal control agreements. Consistency inference and priority propagation are performed on the relationship graph to generate a tag logical relationship strategy. This strategy is a set of executable condition-adjustment rules, recording the condition set, target tag, adjustment direction, and priority, along with a conflict expectation indicator. The tag conflict verification operation reads the tag logical relationship strategy and performs condition matching and conflict detection on the current tag combination. Conflicts include mutually exclusive pairs, level inversion, and cross-dimensional inconsistencies. For each conflict instance, a conflict verification result is provided, including the conflict type, hit conditions, suggested adjustments, and weighting. The final label adjustment process takes the conflict verification results as input, applies the adjustments sequentially according to policy priority and basis weight, and triggers secondary consistency checks and the minimum modification principle when necessary to ensure that the adjustment scope is limited to the minimum number of labels and the shortest time window. The output is a structured label set, which contains the final values ​​of five types of labels, decision links, time intervals and credibility matrices, so that subsequent business systems can directly consume them.

[0154] The sources of call type tags are direct or implicit indicators and guiding sentences of communication purpose in semantic expression, typically implemented through multi-channel lexical matching and integrated voting of intent classifiers; the sources of core appeal tags are the binding relationship between verb-object structures and target objects in knowledge graphs, typically implemented through hybrid parsing of rule templates and lightweight classifiers; the sources of emotion state tags are the type and intensity time series in emotion analysis results, typically implemented through peak preservation and multi-scale smoothing; the sources of urgency level tags are the synthesis of anomaly level and processing timeliness signals, typically implemented through dynamic thresholds and piecewise linear boundaries; the sources of sensitivity level tags are the comparison between sensitive entity categories and compliance rules, typically implemented through entity category mapping and scenario weighting. The sources of tag logical relationship strategies are the fusion of historical statistics, process dependencies, and compliance constraints, typically implemented through relationship graph reasoning and priority propagation; the sources of conflict verification results are the matching of strategy conditions and current tag combinations, typically implemented through condition triggers and conflict type classifiers. All of the above steps use the timeline and speaker dimension of a unified data view as anchor points to ensure that cross-module results can be consistently referenced and verified.

[0155] Example Explanation: In remote health consultation and chronic disease management scenarios within the healthcare field, audio interaction data is widely used to analyze patient needs and psychological states to provide personalized support for health services. When acquiring target audio, the system first collects voice data from the patient's remote voice interaction platform. This data includes patient inquiries regarding dietary advice, chronic disease medication, or rehabilitation progress. The audio preprocessing stage performs noise suppression to remove environmental noise and equipment interference, ensuring clear and reliable voice content. Subsequently, speaker separation distinguishes the patient's and health consultant's voices and segments them into audio segments, ensuring that different dialogue fragments can be analyzed independently. Effective segments that pass audio quality testing are retained and marked, forming the foundational audio data for subsequent processing.

[0156] After receiving the processed audio data, the system loads a customized speech recognition model to transcribe the speech. During warm-up, this model is infused with a vocabulary of medical and health-related terms, such as "blood glucose monitoring," "rehabilitation training," and "dietary intervention," to ensure accurate recognition of relevant content. The transcription results are also corrected for terminology to prevent the mistranslation of proper nouns like "glycated hemoglobin." Simultaneously, the system extracts acoustic features such as fundamental frequency, energy, and speech rate from the audio signal and fuses these features into a consistent acoustic representation using a scale-unified method for subsequent emotion recognition.

[0157] In the emotion analysis phase, acoustic features and transcribed text are processed simultaneously. The acoustic features are aligned to form vector representations, while the text undergoes emotional semantic analysis to generate emotion vectors. The two are then weighted using an attention mechanism and fused into a feature matrix. This matrix is ​​used for complex emotion recognition, outputting the type and intensity of the patient's emotions during the consultation process. For example, when a patient's speech is accelerated or the text contains negative expressions, the system may identify an anxious or worried emotional state. This result provides important support for understanding the patient's psychological state.

[0158] Based on the transcribed text, the system further identifies intent and requests. Through an intent classification model, it can distinguish whether the patient wants medication guidance or inquires about recovery progress. Named entity recognition extracts key information, such as drug name, numerical parameters (blood glucose level), and event type (follow-up appointment time), generating an initial set of elements. Combined with a domain knowledge graph, fuzzy expressions are parsed into standardized intent categories and element sets; for example, "too much medication" can be parsed as a request to "adjust prescription dosage." Subsequently, the system performs multi-round intent tracking, associating elements involved in different rounds of dialogue to form a cross-temporal request chain, ultimately generating structured intent and request identification results.

[0159] In the knowledge base enhancement and matching phase, sentiment analysis results and intent / appeal identification results are fed into the knowledge base for joint processing. The system scans business elements, uses a sensitive entity dictionary to identify personal health information, such as genetic testing results or chronic disease indicators, and generates a set of sensitive tags. Based on the domain knowledge graph, patient appeals are mapped to health management strategies, such as dietary interventions and exercise prescriptions. A regulatory clause library is used to verify the compliance of sentiment and health guidance, avoiding giving high-risk advice when emotions are unstable. Through these processes, anomaly level analysis is generated to identify which appeals require urgent intervention. Finally, all information is integrated to generate knowledge base enhancement results, providing data support for subsequent decision-making.

[0160] Finally, the system generates a set of tags based on the enhanced knowledge base results. Through call type mapping, it determines whether the interaction belongs to health consultation or psychological counseling; through business request analysis, it extracts core request tags, such as "blood glucose monitoring" or "exercise plan"; through emotional state recognition, it generates emotion tags reflecting the patient's psychological state; urgency level classification combines historical abnormality levels with current emotions to indicate whether the problem requires immediate attention; and sensitivity level classification assigns high-sensitivity tags to content involving privacy and important health data. All tags undergo correlation analysis to form logical relationship strategies; for example, the combination of anxiety and high blood glucose levels will increase the urgency level. After tag conflicts are detected and resolved, a final structured tag set is generated for prioritizing health services and providing intervention recommendations.

[0161] In fintech customer interaction and intelligent risk control scenarios, audio data is used to analyze customer behaviors such as reporting incidents, making inquiries, interpreting contract terms, and checking claim progress, in order to provide more efficient support for financial services and risk management. The system first acquires audio data from customer service hotlines or mobile voice interaction channels. This audio may contain dialogue information from multiple roles. To ensure the effectiveness of the analysis, the audio data undergoes preprocessing, including noise suppression to eliminate environmental interference, speaker separation to distinguish between customers and customer service representatives, speech segmentation to form short audio clips that are easy to process, and audio quality detection to filter out distorted or incomplete segments, ultimately generating valid audio data for subsequent processing.

[0162] After receiving the processed audio data, the system loads a customized speech recognition model with an expanded financial vocabulary. During initialization, the model is injected with terms such as "deductible," "claim application," "subrogation," and "policy termination clause" to ensure accurate transcription of key financial concepts. During transcription, terminology correction is performed if the recognized text contains errors; for example, it automatically corrects instances where "loss rate" is misidentified as "payment rate." Simultaneously, the system extracts acoustic features from the audio signal, including fundamental frequency parameters, energy features, and speech rate features, and integrates them into a consistent feature vector using a scaling method. This facilitates subsequent analysis in conjunction with text semantics.

[0163] In the sentiment analysis phase, acoustic features and transcribed text are aligned and vectorized. After sentiment semantic modeling, the text generates sentiment features, while the acoustic features are weighted through an attention mechanism and fused with the text features into a unified matrix. Based on this matrix, the system can identify the customer's emotional state during a call, such as anger, anxiety, dissatisfaction, or calmness, as well as the corresponding emotional intensity. In financial scenarios, this result helps identify potential complaints and high-risk customer emotional fluctuations, supporting subsequent quality control and service optimization.

[0164] In the intent and request identification phase, the system identifies the customer's current goal through an intent classification model, such as "claims reimbursement," "loan consultation," or "policy surrender." Through named entity recognition, it extracts business category entities (e.g., insurance type), numerical parameter entities (e.g., amount, term), and event type entities (e.g., application, change) from the text, forming an initial set of elements. Combined with a knowledge graph from the financial field, fuzzy expressions are parsed into standardized expressions; for example, "want to withdraw money" is parsed as "early withdrawal of policy cash value." A further multi-round tracking mechanism links the customer's requests across multiple interactions. For example, if a customer first requests a claims consultation and then adds "the amount is too small" in a subsequent statement, the two are merged into a complete request across rounds, ultimately generating a structured intent and request identification result.

[0165] When sentiment analysis and intent recognition results are fed into the knowledge base for enhanced matching, the system scans the business elements and identifies highly sensitive data in customer information, such as account numbers and policy numbers, using a sensitive entity dictionary to generate a set of sensitive tags. Based on the knowledge graph, customer requests are mapped to financial business strategies, such as claims review, policy cancellation processes, and loan approvals. A regulatory clause library is invoked to verify the match between emotional states and financial compliance requirements; for example, if a customer requests expedited claims processing in an angry state, the system will mark it as requiring additional manual review. Further anomaly level analysis integrates sensitive entities, business strategies, and compliance verification results to assign a risk level, such as identifying high-urgency complaints. Ultimately, these results are merged to form the knowledge base-enhanced results.

[0166] During the tag generation phase, the system generates multi-dimensional tags based on the enhancement results. Call type mapping generates tags for reporting an incident, claims processing, or inquiries; business request analysis generates core request tags, such as policy cancellation or claims progress; emotional state recognition outputs tags for anger, anxiety, or calmness; urgency level classification combined with dynamic thresholds generates high, medium, and low tags; sensitivity level classification combined with compliance rules outputs general, sensitive, or core tags. These five tag categories undergo correlation analysis to form logical rules; for example, "high sensitivity + high urgency + anger" triggers a manual intervention strategy. If there are conflicts between tags, such as text indicating an inquiry but voice expressing a complaint, the system performs conflict verification and adjusts the tags, ultimately generating a structured tag set. This set can be directly used for customer service quality inspection, claims priority determination, risk warning, or customer value assessment.

[0167] This embodiment uses the knowledge base enhancement results as a unified input to first generate five basic tags, then forms a tag logical relationship strategy based on the relationship graph, and drives the minimal tag adjustment with conflict verification results. This enables the fusion of semantic clues, process clues, and compliance clues into a consistent, traceable, and actionable structured tag set within the same timeline and session. This directly supports priority routing, manual intervention triggering, and risk control, reducing misjudgments caused by terminological ambiguity and cross-dimensional contradictions. The introduction of dynamic thresholds and relationship strategies allows urgency and sensitivity levels to adaptively update with context and handling knowledge, improving the tags' relevance to real business scenarios and their ability to capture high-risk windows.

[0168] In one embodiment, an emotion recognition-based tag generation device is provided, which corresponds one-to-one with the emotion recognition-based tag generation method described in the above embodiments. (Refer to...) Figure 3 , Figure 3This is a schematic diagram of the functional modules of a preferred embodiment of the emotion recognition-based tag generation device of the present invention. The modules include an audio preprocessing module 10, a speech transcription and feature extraction module 20, an emotion analysis module 30, an intent recognition module 40, a knowledge base enhanced matching module 50, and a tag generation module 60. Detailed descriptions of each functional module are as follows:

[0169] The audio preprocessing module 10 is used to acquire the target audio and perform audio preprocessing to obtain the processed audio data;

[0170] The speech transcription and feature extraction module 20 is used to perform speech transcription operations based on the processed audio data, generate transcribed text, and extract acoustic features based on the processed audio data.

[0171] The sentiment analysis module 30 is used to perform sentiment analysis on the acoustic features and the transcribed text, and generate sentiment analysis results.

[0172] The intent recognition module 40 is used to perform intent and appeal recognition based on the transcribed text and extract a set of elements to generate intent and appeal recognition results;

[0173] The knowledge base enhancement matching module 50 is used to perform enhancement matching between the sentiment analysis results and the intent and appeal recognition results within the knowledge base to obtain the knowledge base enhancement results;

[0174] The tag generation module 60 is used to generate a set of tags including call type, core demand, emotional state, urgency level and sensitivity level based on the knowledge base enhancement results.

[0175] In one embodiment, the audio preprocessing module 10 is specifically used for:

[0176] Obtain the target audio;

[0177] Perform noise suppression on the target audio to generate noise-reduced audio data;

[0178] Perform speaker separation on the noise-reduced audio data to generate separated audio segments;

[0179] Perform speech segmentation on the separated audio segments to generate segmented audio data;

[0180] An audio quality detection operation is performed on the segmented audio data. Segmentation marks are added to the segmented audio data that pass the audio quality detection, and processed audio data is generated.

[0181] In one embodiment, the speech transcription and feature extraction module 20 is specifically used for:

[0182] Load a custom speech recognition model, perform a warm-up initialization operation on the custom speech recognition model, and inject an industry terminology vocabulary into the custom speech recognition model;

[0183] The customized speech recognition model is used to perform speech-to-text transcription on the processed audio data, and the speech-to-text transcription results are processed for terminology correction to generate transcribed text.

[0184] The fundamental frequency feature parameters, energy feature parameters, and speech rate feature parameters are extracted from the processed audio data;

[0185] A scaling operation is performed on the fundamental frequency feature parameters, energy feature parameters, and speech rate feature parameters. The scale-scaled fundamental frequency feature parameters, energy feature parameters, and speech rate feature parameters are then fused to generate acoustic features.

[0186] In one embodiment, the sentiment analysis module 30 is specifically used for:

[0187] Perform feature alignment processing on the acoustic features to generate aligned acoustic feature vectors;

[0188] Perform sentiment semantic analysis on the transcribed text to generate a text sentiment feature vector;

[0189] Feature weights are assigned to the aligned acoustic feature vector and the text sentiment feature vector through an attention mechanism to generate a weighted acoustic feature vector and a weighted text feature vector;

[0190] The weighted acoustic feature vector and the weighted text feature vector are fused to generate a fused feature matrix;

[0191] Perform a composite emotion recognition operation on the fused feature matrix to obtain the basic emotion type and emotion intensity;

[0192] The basic emotion types and emotion intensities are combined to generate emotion analysis results.

[0193] In one embodiment, the intent recognition module 40 is specifically used for:

[0194] Perform an intent classification operation on the transcribed text to generate initial intent categories;

[0195] Perform named entity recognition operation on the transcribed text to extract business category entities, numerical parameter entities, and event type entities to generate an initial element set;

[0196] Based on the domain knowledge graph, a fuzzy representation parsing operation is performed on the initial intent category and the initial element set to generate a corrected intent category and a corrected element set, respectively.

[0197] Perform multi-round intent tracking operations on the set of corrected elements, associate historical interaction element relationships, and generate a cross-round associated element set;

[0198] By combining the aforementioned correction intent category and cross-round associated element set, an intent and appeal identification result is generated.

[0199] In one embodiment, the knowledge base enhancement matching module 50 is specifically used for:

[0200] Scan the business elements in the intent and request identification results, identify the personal identity information entities in the business elements through a sensitive entity dictionary, and generate a sensitive entity tag set;

[0201] Based on the domain knowledge graph, a business strategy mapping operation is performed on the intent and request identification results to generate business strategy mapping results;

[0202] Based on the regulatory provisions library, the emotion analysis results are subjected to emotion compliance verification to detect conflicts between the emotional state and regulatory requirements, and an emotion compliance tag is generated.

[0203] Based on the sensitive entity tag set, business strategy mapping results, and sentiment compliance tags, perform an anomaly level analysis operation to generate an anomaly level analysis result.

[0204] By integrating the sensitive entity tag set, business strategy mapping results, sentiment compliance tags, and anomaly level analysis results, an enhanced knowledge base result is generated.

[0205] In one embodiment, the label generation module 60 is specifically used for:

[0206] Perform a call type mapping operation on the knowledge base enhancement results to generate call type tags;

[0207] Perform business request parsing operations on the knowledge base enhancement results to generate core request tags;

[0208] Perform an emotion state recognition operation on the knowledge base enhancement results to generate emotion state labels;

[0209] Based on a dynamic threshold, the knowledge base enhancement results are subjected to an urgency level classification operation to generate urgency level labels;

[0210] Based on the compliance policy, perform a sensitivity level classification operation on the knowledge base enhancement results to generate sensitivity level labels;

[0211] Perform tag association analysis on the call type tag, core demand tag, emotional state tag, urgency tag, and sensitivity level tag to establish a tag logical relationship strategy;

[0212] Based on the aforementioned tag logical relationship strategy, a tag conflict verification operation is performed to generate a conflict verification result.

[0213] The conflict verification results are used to adjust the call type label, core demand label, emotional state label, urgency label, and sensitivity level label to generate a structured label set.

[0214] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When executed by the processor, the computer program implements the functions or steps of a server-side method for emotion recognition-based tag generation.

[0215] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides determination and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When executed by the processor, the computer program implements client-side functions or steps of an emotion recognition-based tag generation method.

[0216] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0217] The target audio is acquired and audio preprocessing is performed to obtain the processed audio data;

[0218] Based on the processed audio data, a speech-to-text operation is performed to generate transcribed text, and acoustic features are extracted based on the processed audio data.

[0219] Sentiment analysis is performed on the acoustic features and the transcribed text to generate sentiment analysis results.

[0220] Based on the transcribed text, the intent and appeal are identified and a set of elements are extracted to generate the intent and appeal identification result.

[0221] The emotion analysis results and the intent and appeal recognition results are subjected to enhanced matching within the knowledge base to obtain the knowledge base enhanced results.

[0222] Based on the enhanced results of the knowledge base, a set of tags including emotional state, urgency level, and sensitivity level is generated.

[0223] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0224] The target audio is acquired and audio preprocessing is performed to obtain the processed audio data;

[0225] Based on the processed audio data, a speech-to-text operation is performed to generate transcribed text, and acoustic features are extracted based on the processed audio data.

[0226] Sentiment analysis is performed on the acoustic features and the transcribed text to generate sentiment analysis results.

[0227] Based on the transcribed text, the intent and appeal are identified and a set of elements are extracted to generate the intent and appeal identification result.

[0228] The emotion analysis results and the intent and appeal recognition results are subjected to enhanced matching within the knowledge base to obtain the knowledge base enhanced results.

[0229] Based on the enhanced results of the knowledge base, a set of tags including emotional state, urgency level, and sensitivity level is generated.

[0230] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0231] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0232] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0233] It should be noted that if any software tools or components not belonging to this company appear in the embodiments of this application, they are merely illustrative examples and do not represent actual use. The embodiments described above are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for generating a tag based on emotion recognition, the method comprising: The method comprises the following steps: obtaining target audio and performing audio preprocessing to obtain processed audio data; performing speech transcription based on the processed audio data to generate a transcription text, and extracting acoustic features based on the processed audio data; performing emotion analysis on the acoustic features and the transcription text to generate an emotion analysis result; performing intent and appeal recognition based on the transcription text and extracting an element set to generate an intent and appeal recognition result; performing enhanced matching on the emotion analysis result and the intent and appeal recognition result in a knowledge base to obtain a knowledge base enhancement result; generating a label set comprising an emotional state, an urgency level and a sensitivity level according to the knowledge base enhancement result. 2.The emotion recognition-based tag generation method of claim 1, wherein, obtaining target audio and performing audio preprocessing to obtain processed audio data, comprising: obtaining target audio; performing noise suppression on the target audio to generate noise-reduced audio data; performing speaker separation on the noise-reduced audio data to generate separated audio segments; performing speech segmentation on the separated audio segments to generate segmented audio data; performing audio quality detection on the segmented audio data, adding a segmentation mark to the segmented audio data that passes the audio quality detection, and generating processed audio data. 3.The emotion recognition-based tag generation method of claim 1, wherein, performing speech transcription based on the processed audio data to generate a transcription text, and extracting acoustic features based on the processed audio data, comprising: loading a customized speech recognition model, performing pre-warming initialization on the customized speech recognition model, and injecting an industry terminology table into the customized speech recognition model; performing speech transcription on the processed audio data by the customized speech recognition model, and performing terminology correction processing on the speech transcription result to generate a transcription text; extracting fundamental frequency feature parameters, energy feature parameters and speech rate feature parameters from the processed audio data; performing scale unification on the fundamental frequency feature parameters, energy feature parameters and speech rate feature parameters, fusing the scale-unified fundamental frequency feature parameters, energy feature parameters and speech rate feature parameters, and generating acoustic features. 4.The emotion recognition-based tag generation method of claim 1, wherein, performing emotion analysis on the acoustic features and the transcription text to generate an emotion analysis result, comprising: performing feature alignment processing on the acoustic features to generate aligned acoustic feature vectors; performing sentiment semantic analysis on the transcription text to generate text sentiment feature vectors; assigning feature weights to the aligned acoustic feature vectors and text sentiment feature vectors through an attention mechanism to generate weighted acoustic feature vectors and weighted text feature vectors; fusing the weighted acoustic feature vectors and weighted text feature vectors to generate a fusion feature matrix; performing complex emotion recognition on the fusion feature matrix to obtain a basic emotion type and an emotion intensity; combining the basic emotion type and the emotion intensity to generate an emotion analysis result. 5.The emotion recognition-based tag generation method of claim 1, wherein, performing intent and appeal recognition based on the transcription text and extracting an element set to generate an intent and appeal recognition result, comprising: performing intent classification on the transcription text to generate an initial intent category; Perform a named entity recognition operation on the transcribed text, extract business category entities, numerical parameter entities, and event type entities, and generate an initial element set; Perform a fuzzy expression analysis operation on the initial intent category and initial element set based on the domain knowledge graph, and generate a corrected intent category and a corrected element set, respectively; Perform a multi-round intent tracking operation on the corrected element set, associate historical interaction element relationships, and generate a cross-round associated element set; Combine the corrected intent category and the cross-round associated element set to generate an intent and appeal recognition result. 6.The emotion recognition-based tag generation method of claim 1, wherein, Perform enhanced matching of the emotion analysis result and the intent and appeal recognition result in the knowledge base to obtain a knowledge base enhancement result, including: Scan the business elements in the intent and appeal recognition result, identify personal identity information entities in the business elements through a sensitive entity dictionary, and generate a sensitive entity label set; Perform a business strategy mapping operation on the intent and appeal recognition result based on the domain knowledge graph to generate a business strategy mapping result; Perform a sentiment compliance verification operation on the emotion analysis result according to the regulatory clause library to detect conflicts between emotion states and regulatory requirements, and generate a sentiment compliance label; Perform an abnormal level analysis operation based on the sensitive entity label set, business strategy mapping result, and sentiment compliance label to generate an abnormal level analysis result; Fuse the sensitive entity label set, business strategy mapping result, sentiment compliance label, and abnormal level analysis result to generate a knowledge base enhancement result. 7.The emotion recognition-based tag generation method of claim 1, wherein, Generate a label set including emotion states, urgency levels, and sensitivity levels based on the knowledge base enhancement result, including: Perform a call type mapping operation on the knowledge base enhancement result to generate a call type label; Perform a business appeal analysis operation on the knowledge base enhancement result to generate a core appeal label; Perform an emotion state recognition operation on the knowledge base enhancement result to generate an emotion state label; Perform an urgency level classification operation on the knowledge base enhancement result based on a dynamic threshold to generate an urgency level label; Perform a sensitivity level classification operation on the knowledge base enhancement result based on compliance strategies to generate a sensitivity level label; Perform a label association analysis operation on the call type label, core appeal label, emotion state label, urgency level label, and sensitivity level label to establish a label logical relationship strategy; Perform a label conflict verification operation based on the label logical relationship strategy to generate a conflict verification result; Adjust the call type label, core appeal label, emotion state label, urgency level label, and sensitivity level label through the conflict verification result to generate a structured label set. 8.A label generation apparatus based on emotion recognition, characterized by, The emotion recognition-based label generation apparatus includes: An audio preprocessing module configured to obtain a target audio and perform audio preprocessing to obtain processed audio data; A speech transcription and feature extraction module configured to perform a speech transcription operation based on the processed audio data to generate transcribed text, and extract acoustic features based on the processed audio data; An emotion analysis module configured to perform emotion analysis on the acoustic features and the transcribed text to generate an emotion analysis result; An intent recognition module is configured to perform intent and appeal recognition and extract a set of elements based on the transcribed text, and generate an intent and appeal recognition result; A knowledge base enhanced matching module is configured to perform enhanced matching of the emotion analysis result and the intent and appeal recognition result in a knowledge base, and obtain a knowledge base enhanced result; A label generation module is configured to generate a set of labels including a call type, a core appeal, an emotion state, an emergency level, and a sensitivity level according to the knowledge base enhanced result.

9. A computer device, comprising: The computer device includes a memory, a processor, and a label generation program based on emotion recognition stored on the memory and executable on the processor, and the label generation program based on emotion recognition, when executed by the processor, implements the steps of the label generation method based on emotion recognition in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium has a label generation program based on emotion recognition stored thereon, and the label generation program based on emotion recognition, when executed by the processor, implements the steps of the label generation method based on emotion recognition in any one of claims 1-7.

Images