Intelligent management method and device for customer complaints and electronic equipment

Abstract

The application relates to the field of customer complaint management, and specifically discloses a customer complaint intelligent management method and device and electronic equipment. First, customer complaint text data, customer complaint audio data and customer complaint picture data collected by a complaint platform are acquired, then deep learning technology is used to perform feature extraction and correlation analysis on the three, and finally, a classifier is used to obtain customer complaint problem classification labels, so that more comprehensive and accurate problem identification is realized, and then the enterprise is helped to quickly respond, improve products or services, optimize the complaint management process, and improve customer satisfaction and loyalty.

Description

Technical Field

[0001] This application relates to the field of customer complaint management, and more specifically, to an intelligent management method, apparatus, and electronic device for customer complaints. Background Technology

[0002] To improve product quality, businesses often provide customers with a dedicated complaint channel. Customer complaints may include issues such as product defects, poor service, or delivery delays. Effective customer complaint management can not only help businesses improve their products and services but also enhance customer satisfaction and loyalty.

[0003] Traditional customer complaint handling methods rely heavily on manual screening and classification. Since manual screening and classification depend on the experience of the screening personnel to analyze the reasons for complaints and pinpoint the root causes, this approach is not only inefficient but also prone to errors. Manual screening and classification often struggles to process large volumes of complaint information quickly and accurately, potentially leading to missed issues or delayed responses, thus impacting the company's responsiveness and improvement efforts.

[0004] Therefore, there is a need for an intelligent management method, device, and electronic equipment for customer complaints. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide an intelligent customer complaint management method, apparatus, and electronic device. It first acquires customer complaint text data, customer complaint audio data, and customer complaint image data collected by a complaint platform. Then, it utilizes deep learning technology to extract features and perform correlation analysis on the three data sets. Finally, it uses a classifier to obtain customer complaint problem classification labels, thereby achieving more comprehensive and accurate problem identification. This helps enterprises respond quickly, improve products or services, optimize complaint management processes, and enhance customer satisfaction and loyalty.

[0006] According to one aspect of this application, an intelligent management method for customer complaints is provided, comprising:

[0007] Acquire customer complaint text data, customer complaint audio data, and customer complaint image data collected by the complaint platform;

[0008] Extract semantic feature vectors of customer complaint text content and multimodal association feature vectors of customer complaints from the customer complaint text data, customer complaint audio data and customer complaint image data collected by the complaint platform;

[0009] Based on the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint, a classification label for the customer complaint issue is obtained.

[0010] According to another aspect of this application, an intelligent customer complaint management device is provided, comprising:

[0011] The complaint platform data acquisition module is used to acquire customer complaint text data, customer complaint audio data, and customer complaint image data collected by the complaint platform.

[0012] The complaint platform data extraction module is used to extract semantic feature vectors of customer complaint text content and multimodal association feature vectors of customer complaints from the customer complaint text data, customer complaint audio data and customer complaint image data collected by the complaint platform;

[0013] The customer complaint issue label classification module is used to obtain customer complaint issue classification labels based on the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint.

[0014] According to another aspect of this application, an electronic device is provided, comprising: a processor; and a memory storing computer program instructions, which, when executed by the processor, cause the processor to perform the intelligent customer complaint management method as described above.

[0015] Compared with existing technologies, the intelligent management method, device, and electronic equipment for customer complaints provided in this application first acquire customer complaint text data, customer complaint audio data, and customer complaint image data collected by the complaint platform. Then, deep learning technology is used to extract features and perform correlation analysis on the three data. Finally, a classifier is used to obtain customer complaint problem classification labels, thereby achieving more comprehensive and accurate problem identification. This helps enterprises respond quickly, improve products or services, optimize complaint management processes, and enhance customer satisfaction and loyalty. Attached Figure Description

[0016] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0017] Figure 1 This is a flowchart of an intelligent customer complaint management method according to an embodiment of this application.

[0018] Figure 2 This is a flowchart illustrating the process of extracting features from customer complaint text data to obtain semantic feature vectors of the customer complaint text content in the intelligent customer complaint management method according to an embodiment of this application.

[0019] Figure 3 This is a flowchart illustrating the process of extracting features from the audio data of customer complaints to obtain a global feature vector of the speech spectrum of customer complaints in the intelligent management method for customer complaints according to an embodiment of this application.

[0020] Figure 4 This is a flowchart illustrating the process of extracting features from customer complaint image data to obtain a feature vector of customer complaint image information in the intelligent management method for customer complaints according to an embodiment of this application.

[0021] Figure 5 This is a block diagram of an intelligent customer complaint management device according to an embodiment of this application. Detailed Implementation

[0022] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0023] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0024] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed description. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0025] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0026] Figure 1 This is a flowchart of an intelligent customer complaint management method according to an embodiment of this application. Figure 1As shown, the intelligent customer complaint management method according to an embodiment of this application includes: S110, acquiring customer complaint text data, customer complaint audio data, and customer complaint image data collected by a complaint platform; S120, extracting customer complaint text content semantic feature vectors and customer complaint multimodal association feature vectors from the customer complaint text data, customer complaint audio data, and customer complaint image data collected by the complaint platform; S130, obtaining customer complaint problem classification labels based on the customer complaint text content semantic feature vectors and the customer complaint multimodal association feature vectors.

[0027] In the aforementioned intelligent customer complaint management method, step S110 involves acquiring customer complaint text data, audio data, and image data collected by the complaint platform. It should be understood that effective complaint management can not only help businesses improve their products and services but also enhance customer satisfaction and loyalty. Merchants typically establish dedicated complaint channels to improve product quality. Customer complaints may involve issues such as product defects, poor service attitude, or delivery delays. However, traditional complaint handling methods mainly rely on manual screening and classification. These methods depend on the experience of the screening personnel to analyze the reasons for complaints, which is usually inefficient and error-prone, making it difficult to quickly and accurately process large amounts of complaint information. This may lead to missed issues or processing delays, thereby affecting the company's response to problems and improvement effects. Therefore, in the technical solution of this application, by acquiring customer complaint text data, audio data, and image data collected by the complaint platform and combining them with deep learning technology, customer complaint problem classification labels are obtained to achieve more comprehensive and accurate problem identification. This helps companies respond quickly, improve products or services, optimize complaint handling processes, and ultimately enhance customer satisfaction and loyalty.

[0028] Specifically, acquiring multimodal data—including text, audio, and image data of customer complaints collected by complaint platforms—provides a multi-layered perspective on customer complaints, enabling deeper analysis and problem-solving. Text data typically includes written descriptions submitted by customers through online forms, emails, or social media, directly reflecting the details, emotions, and specific issues of the complaint. Audio data provides the voice information expressed by customers in phone calls or voicemails, capturing tone, intonation, and emotional changes that text data cannot fully convey. Image data may include photos taken by customers of product defects, service environments, or other related issues, visually demonstrating the actual situation and facilitating a clearer understanding of the complaint. By integrating this multimodal data, businesses can obtain more comprehensive complaint information, allowing for the comprehensive consideration of different types of evidence during analysis, improving the accuracy and efficiency of complaint handling. This intelligent management approach helps automate the processing of large volumes of complaint data, reducing errors and delays caused by human intervention, ultimately improving customer satisfaction and the company's responsiveness.

[0029] In the aforementioned intelligent customer complaint management method, step S120 involves extracting semantic feature vectors of the customer complaint text data, audio data, and image data collected by the complaint platform, as well as multimodal association feature vectors. It should be understood that a single data source may not fully reflect the complexity of a complaint. By extracting semantic feature vectors of the text content and multimodal association feature vectors, information from different data sources can be combined to form a comprehensive complaint feature model. This model can more accurately capture the key issues and emotional states in a complaint, thereby supporting more efficient classification and processing decisions.

[0030] In a specific embodiment of this application, step S120 includes: extracting features from the customer complaint text data to obtain a semantic feature vector of the customer complaint text content; extracting features from the customer complaint audio data to obtain a global feature vector of the customer complaint speech spectrum; extracting features from the customer complaint image data to obtain a feature vector of the customer complaint image information; and associating the global feature vector of the customer complaint speech spectrum and the feature vector of the customer complaint image information to obtain a multimodal association feature vector of the customer complaint.

[0031] It is understandable that customer complaint texts are typically in natural language, possessing a high degree of freedom and complexity. The text content can include various expressions, grammatical structures, and sentiments, making direct analysis and processing difficult. Feature extraction can transform this complex textual data into unified, structured feature vectors, thus simplifying data processing and analysis and enabling the effective capture and representation of core information and semantic relationships within customer complaint texts.

[0032] Furthermore, raw audio data is unstructured and contains a wealth of information. By extracting global features from the audio data, such as tone, rate of speech, pitch, and intensity, it is possible to more accurately identify the customer's emotional state. For example, a rapid speech rate may reflect a customer's dissatisfaction or tension, while a steady tone may indicate a more neutral attitude. Transforming this emotional information into feature vectors helps to better understand the customer's actual feelings and the urgency of their complaint.

[0033] Furthermore, raw image data typically contains a wealth of information, and directly analyzing this data can present high computational complexity and processing difficulties. Feature extraction allows us to extract key visual features from images, such as color distribution, texture patterns, and shape contours. These feature vectors can effectively represent important information in the image, such as the type of damage to a product or the state of the service environment, helping us focus on crucial visual information when processing and analyzing images.

[0034] In particular, speech and images offer two distinct but complementary sources of information. Single-modal data analysis may miss crucial information. By combining speech and image features, the strengths of these two sources can be leveraged. For example, flaws in an image may require description in speech to confirm their nature, while the emotion in speech may correspond to the type of problem in the image. Multimodal correlation makes the analysis more accurate, reducing misjudgments and omissions, and ensuring a comprehensive identification of complaints.

[0035] Figure 2 This is a flowchart illustrating the process of extracting features from customer complaint text data to obtain a semantic feature vector of the customer complaint text content in the intelligent customer complaint management method according to an embodiment of this application. Figure 2 As shown, in a specific embodiment of this application, feature extraction is performed on the customer complaint text data to obtain the semantic feature vector of the customer complaint text content, including: S210, data preprocessing is performed on the customer complaint text data to obtain a sequence of word embedding vectors for the customer complaint text content; S220, the sequence of word embedding vectors for the customer complaint text content is passed through a converter-based customer complaint text content semantic encoder model to obtain the semantic feature vector of the customer complaint text content.

[0036] It is understandable that raw text data often contains noise, such as spelling errors, punctuation, stop words, and unnecessary whitespace. This noise can interfere with text analysis and reduce the accuracy of the model. Data preprocessing, by standardizing text (e.g., converting to lowercase, removing punctuation) and cleaning up irrelevant information, helps ensure the consistency and reliability of text data, making subsequent analysis more accurate. Word embedding techniques (such as Word2Vec, GloVe, or BERT) transform words in text into vectors that capture the semantic information of words in a high-dimensional space. Sequences of word embedding vectors can map semantically similar words to nearby vector space positions, thus preserving the semantic relationships between words. This representation method allows machine learning models to effectively understand and process text data, identifying potential problems and sentiment tendencies in customer complaints. Through data preprocessing and word embedding generation, the contextual information of the text is preserved and transformed into a vector representation. For example, word embeddings can not only represent the semantics of words but also reflect the meaning of words in a specific context. Processing customer complaint texts helps models capture the meaning of words in specific complaint scenarios, improving the ability to understand and analyze customer complaint content.

[0037] Furthermore, while traditional word embedding methods such as Word2Vec and GloVe can effectively convert words into vectors, they typically only consider the local context of the words. Semantic encoder models based on converters (such as BERT or GPT), through self-attention mechanisms, can globally capture the complex relationships and contextual information between words in the text. This means that the representation of each word is not only related to itself but also considers the context of the entire sentence or paragraph, thus reflecting its semantics more accurately. Specifically, the converter-based BERT model of the aforementioned converter-based customer complaint text content semantic encoder model is used to perform global contextual semantic encoding on the sequence of word embedding vectors of the customer complaint text content to obtain multiple customer complaint feature vectors; and the multiple customer complaint feature vectors are concatenated to obtain the semantic feature vector of the customer complaint text content.

[0038] Figure 3 This is a flowchart illustrating the process of extracting features from the audio data of customer complaints to obtain a global feature vector of the customer complaint speech spectrum in the intelligent customer complaint management method according to an embodiment of this application. Figure 3As shown, in a specific embodiment of this application, feature extraction is performed on the customer complaint audio data to obtain a global feature vector of the customer complaint speech spectrum, including: S310, performing feature preprocessing on the customer complaint audio data to obtain a multi-channel speech spectrogram of the customer complaint; S320, passing the multi-channel speech spectrogram of the customer complaint through a convolutional neural network based on a two-stream network model to obtain the global feature vector of the customer complaint speech spectrum.

[0039] It's understandable that raw audio data is a continuous waveform signal, which is difficult for computers to process and analyze directly. Feature preprocessing transforms the audio signal into a spectrogram, a visual representation of the frequency components of a sound signal distributed along a time axis. Multi-channel speech spectrograms further utilize information from multiple spectral channels, providing more detailed speech features that help models better understand and analyze audio data. Specifically, multi-channel speech spectrograms can capture sound signal features from multiple perspectives, such as frequency, amplitude, and phase information. By extracting the audio signal using Short-Time Fourier Transform (STFT) or Mel-frequency cepstral coefficients (MFCC), the frequency components of each time segment can be obtained. This fine-grained feature representation allows the audio data to more comprehensively reflect key information such as pitch, speech rate, and tone in customer complaints. Multi-channel spectrograms can more accurately capture these variations, improving the model's ability to recognize emotions and intentions.

[0040] Furthermore, multi-channel speech spectrograms process different feature dimensions of the audio signal separately (such as different frequency bandwidths or different time windows), allowing for the extraction of rich local features from each channel. The two-stream network model can simultaneously process and fuse information from different channels, thus comprehensively considering the multi-dimensional features of the audio signal. This fusion helps capture complex patterns and details in the sound signal, improving the model's understanding of complaint content. Specifically, by applying the two-stream network model to multi-channel speech spectrograms, local features in the audio signal, such as pitch, rhythm, and timbre, can be effectively extracted. The combination of these local features provides important information about customer emotions, intonation variations, and other speech characteristics, laying the foundation for subsequent analysis and decision-making. Specifically, the two-stream network model consists of two parallel convolutional neural networks, typically one processing low-level features (such as the original spectrogram) and the other processing high-level features (such as the pre-processed spectrogram). This design improves the model's ability to extract features at different levels, thereby better capturing subtle changes and global information in the audio signal. The two-stream network model combines local features and global information to provide a more accurate and comprehensive representation of audio features.

[0041] In a specific embodiment of this application, step S310 includes: passing the customer complaint audio data through a customer complaint audio information noise reduction module based on an autoencoder to obtain noise-reduced customer complaint audio information; extracting the customer complaint audio logarithmic Melp map, the customer complaint audio cochlear spectrogram, and the customer complaint audio constant Q transform spectrogram from the noise-reduced customer complaint audio information; and arranging the customer complaint audio logarithmic Melp map, the customer complaint audio cochlear spectrogram, and the customer complaint audio constant Q transform spectrogram to form the customer complaint multi-channel speech spectrogram.

[0042] It's understandable that in practical applications, customer complaint audio data is often affected by background noise, environmental interference, or poor recording equipment quality. This noise and interference can mask or distort important information in the audio signal, making speech recognition and sentiment analysis difficult. Noise reduction is a key step in solving this problem; it removes unnecessary noise, making the audio signal cleaner and improving the quality of subsequent processing. An autoencoder is an unsupervised learning model capable of learning efficient representations of data. It consists of an encoder and a decoder. The encoder maps the input data to a low-dimensional latent space, while the decoder reconstructs the original data from this low-dimensional space. In this way, the autoencoder can learn effective features of the data and remove noise during reconstruction. Especially for audio data, the autoencoder can capture the main structure and features of the sound signal during encoding, thus effectively removing noise during decoding. In the autoencoder-based noise reduction module, the encoder transforms the noisy customer complaint audio data into a low-dimensional feature representation. These feature representations retain the core information of the audio signal while removing some noise. The decoder then reconstructs a clear audio signal from these low-dimensional features. Since noise typically lacks a systematic structure, it can be effectively suppressed or removed during reconstruction, thereby improving the clarity of the audio data. Through noise reduction processing, key information in the audio data, such as the content and emotional expression of the speech, can be more clearly preserved and extracted. Specifically, the customer complaint audio data is input into the encoder of the autoencoder-based customer complaint audio information noise reduction module, wherein the encoder uses a convolutional layer to explicitly spatially encode the customer complaint audio data to obtain speech features; and the speech features are input into the decoder of the autoencoder-based customer complaint audio information noise reduction module, wherein the decoder uses a deconvolutional layer to deconvolve the speech features to obtain the noise-reduced customer complaint audio information.

[0043] Furthermore, the main purpose of extracting log-Mel spectrograms, cochlear spectrograms, and constant Q-transform spectrograms from the denoised customer complaint audio information is to comprehensively and effectively analyze the characteristics of the audio data. Each spectrogram type provides different levels of audio information, helping to more accurately understand and process the audio content of customer complaints. Specifically, the log-Mel spectrogram is a commonly used spectrogram type in audio signal processing. It represents the audio signal's spectrum by converting it to a Mel scale (mimicking the human ear's perception of different frequencies). The Mel scale better matches the auditory characteristics of the human ear, amplifying details in the low-frequency range while compressing high-frequency components. The log-Mel spectrogram, through a logarithmic transformation of the Mel spectrum, helps to emphasize the energy distribution in the audio signal and converts it to a logarithmic scale for better handling of the audio's dynamic range. This spectrogram is particularly important for speech recognition and sentiment analysis because it can capture subtle changes in speech and help extract effective features.

[0044] Cochlear spectrograms (or cochlear model spectrograms) simulate the frequency analysis mechanism of the human inner ear cochlea. The cochlea is a biologically sophisticated frequency decomposition system, and by simulating its function, cochlear spectrograms can provide a frequency response that more closely matches the biological auditory system. This type of spectrogram can more accurately simulate and capture the details of audio signals, especially useful for analyzing high-frequency components and subtle signal variations in audio signals. It can help understand and analyze speech details in customer complaints, such as pitch variations and volume fluctuations.

[0045] The constant Q transform spectrum is a spectral analysis method based on a constant Q value, where Q represents the ratio of bandwidth to center frequency. The constant Q transform offers high frequency resolution and good time resolution, providing stable resolution across different frequency ranges. This spectrum is particularly suitable for processing music and complex audio signals because it effectively captures the spectral characteristics of the audio signal and provides balanced analysis across different frequency bands. For customer complaint audio, this spectrum can help analyze details in the audio signal, such as the spectral characteristics of speech and potential background noise, thereby further improving the ability to understand and analyze customer speech.

[0046] By extracting log-Mel spectrum, cochlear spectrum, and constant Q-transform spectrum, multidimensional features of audio signals can be obtained. These features provide comprehensive information about audio signals at different frequency scales and perceptual models, thereby enhancing the analytical capabilities of audio data.

[0047] Furthermore, in the technical solution of this application, each spectrogram type—log-Mel spectrogram, cochlear spectrogram, and constant Q-transform spectrogram—provides different perspectives for spectral analysis. Specifically, the log-Mel spectrogram, by simulating the auditory characteristics of the human ear, can effectively handle the dynamic range and energy distribution in audio signals; the cochlear spectrogram simulates the biological frequency decomposition mechanism of the cochlea, enabling the capture of details and subtle variations in audio signals; and the constant Q-transform spectrogram provides high-frequency resolution and balanced spectral analysis. By arranging these three spectrograms into a multi-channel speech spectrogram, the information contained in each can be integrated to form a multi-dimensional feature representation. This multi-channel representation provides detailed information about the audio signal at different frequencies and time scales, thereby enhancing the information extraction capability and more comprehensively capturing key features in the audio, such as the spectral characteristics of speech, pitch variations, and background noise, thus improving the accuracy of the analysis.

[0048] Figure 4 This is a flowchart illustrating the process of extracting features from customer complaint image data to obtain a customer complaint image information feature vector in the intelligent customer complaint management method according to an embodiment of this application. Figure 4 As shown, in a specific embodiment of this application, feature extraction is performed on the customer complaint image data to obtain a customer complaint image information feature vector, including: S410, passing the customer complaint image data through a customer complaint image information feature encoder based on a deep neural network to obtain a customer complaint image information feature map; S420, pooling the customer complaint image information feature map to obtain the customer complaint image information feature vector.

[0049] It is understandable that image data typically has high dimensionality, and directly processing the raw image data can lead to high computational complexity and processing difficulty. Through a deep neural network encoder, image data is transformed into a lower-dimensional feature map. This dimensionality reduction not only reduces the computational burden but also preserves key information in the image. Traditional image processing methods often require manually designing feature extractors, while deep neural networks can automatically learn and extract important features from images through their multi-layered structure. The deep neural network-based image information feature encoder can capture complex patterns and structures in images, such as texture, shape, and color distribution, thereby forming an informative feature map. This efficient feature extraction improves the accuracy and efficiency of subsequent analysis. Specifically, each layer of the deep neural network-based customer complaint image information feature encoder performs convolution processing, mean pooling based on the local feature matrix, and non-linear activation processing on the input data during the forward pass of the layer, so that the final layer of the deep neural network-based customer complaint image information feature encoder outputs the customer complaint image information feature map, where the input of the deep neural network-based customer complaint image information feature encoder is the customer complaint image data.

[0050] Furthermore, pooling the feature map of customer complaint images allows for the extraction of representative and concise feature representations. Pooling, by applying techniques such as max pooling or average pooling to the feature map, effectively reduces its spatial dimensionality. This process compresses the detailed information in the original image into a smaller size while retaining important feature information. Dimensionality reduction reduces computational complexity, lowers storage requirements, and speeds up subsequent processing steps. During pooling, the maximum or average value of a local region in the feature map is typically chosen to represent the information of that region. Max pooling, for example, preserves the most salient features in a local region, ensuring that critical information is not lost in downstream tasks. This helps extract and retain salient features in the image, such as object edges or textures, while ignoring unimportant details and noise. The pooled feature vector is a simplified image representation, compressing the original high-dimensional feature map into a lower-dimensional vector. This simplification makes it easier for subsequent machine learning models, such as classifiers or regressors, to process and analyze these features. Feature vectors contain the core information of an image, but redundant data has been removed, which helps improve the efficiency of model training and prediction.

[0051] In the aforementioned intelligent customer complaint management method, step S130 involves obtaining a customer complaint issue classification label based on the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint. It should be understood that the combination of multimodal data not only improves the comprehensiveness of classification but also enhances the robustness of the model. For example, if the complaint text information is unclear or ambiguous, supplementary details provided by multimodal information (such as images or audio) can help the model better understand the actual situation of the complaint, thereby reducing misclassification caused by insufficient information or noise. Accurate complaint classification helps to handle customer issues more effectively.

[0052] In a specific embodiment of this application, step S130 includes: performing backpropagation displacement information compensation based on semantic space amplitude on the semantic feature vector of customer complaint text content and the multimodal association feature vector of customer complaint to obtain a customer complaint problem classification feature vector; passing the customer complaint problem classification feature vector through a classifier to obtain a classification result, the classification result being used to represent a customer complaint problem classification label.

[0053] It's understandable that customer complaints typically involve multiple information formats, such as text descriptions, image attachments, and audio recordings. A single information source may not be able to fully capture the details and context of a complaint. For example, text can convey specific demands and emotions, but images or audio can provide additional background information. By fusing feature vectors from these different sources, the content of the complaint can be comprehensively acquired and understood, ensuring that no important information is missed. Specifically, the text content semantic feature vector extracts linguistic semantic information from the text data, such as keywords, themes, and sentiment. Meanwhile, the multimodal association feature vector incorporates information from other modalities such as images and audio, providing richer context.

[0054] Specifically, in the technical solution of this application, text feature vectors are typically based on word embedding models, such as Word2Vec or BERT, capturing the semantics and contextual relationships within the text. Audio feature vectors, such as log-Mel spectrograms, cochlear spectrograms, and constant Q-transform spectrograms, primarily focus on the spectral information of sound, while audio processing models often focus on the temporal and frequency domain characteristics of sound. Image feature vectors are extracted by deep convolutional neural networks, mainly focusing on the spatial features and texture information in images. These three modalities have fundamental differences in spatial distribution and representational dimensions, leading to fine-grained dimensional shifts during simple fusion, i.e., inconsistent scales and representations of each feature in the feature space. When features from different sources (text, audio, and images) are integrated, their local structure and semantic information may be diluted or distorted during simple fusion, causing highly specific local features to become invalid. Specifically, textual features may be effective at distinguishing subtle semantic differences in complaint content, but when fused with audio and image features, these fine-grained semantic differences may be denoised or lost, resulting in the final feature vector failing to effectively retain this local information and thus affecting the accuracy of the classification results. Therefore, the technical solution of this application obtains the customer complaint problem classification feature vector by performing backpropagation displacement information compensation based on semantic space amplitude on the semantic feature vector of customer complaint text content and the multimodal association feature vector of customer complaints.

[0055] The process of performing backpropagation displacement information compensation based on semantic space amplitude on the semantic feature vector of customer complaint text content and the multimodal association feature vector of customer complaint to obtain a customer complaint problem classification feature vector includes: calculating the positional difference between the semantic feature vector of customer complaint text content and the multimodal association feature vector of customer complaint to obtain a displacement information compensation difference feature vector; calculating the positional dot product between the semantic feature vector of customer complaint text content and the multimodal association feature vector of customer complaint to obtain a displacement information compensation positional dot product feature vector; calculating the positional summation between the semantic feature vector of customer complaint text content and the multimodal association feature vector of customer complaint to obtain a displacement information compensation positional summation feature vector; and combining the displacement information compensation difference feature vector and the displacement information compensation positional summation feature vector. The dot product feature vector and the displacement information compensation feature vector summed positionally are concatenated and input into a one-dimensional convolutional layer and subjected to max pooling to obtain the displacement information compensation convolutional coding pooled feature vector. The square root of the L2 norm of the displacement information compensation convolutional coding pooled feature vector is weighted and added to the L1 norm of the displacement information compensation convolutional coding pooled feature vector to obtain the displacement information compensation factor. The displacement information compensation dot product feature vector summed positionally is added to the square root of the length of the semantic feature vector of the customer complaint text content positionally to obtain the displacement information compensation intermediate vector. Using the displacement information compensation factor as the first weighting coefficient and the second weighting hyperparameter as the second weighting coefficient, the positionally weighted sum of the displacement information compensation feature vector summed positionally and the displacement information compensation intermediate vector is calculated to obtain the customer complaint problem classification feature vector.

[0056] Specifically, the semantic feature vector of customer complaint text content and the multimodal association feature vector of customer complaints are compensated for by backpropagation displacement information based on semantic space amplitude to obtain the customer complaint issue classification feature vector, including: ; ; ;in, This represents the semantic feature vector of the customer complaint text content. This represents the multimodal correlation feature vector of customer complaints. This indicates subtraction by position. This indicates dot product by position. This indicates addition by position. This represents a one-dimensional convolutional layer. This indicates max pooling. This represents the feature vector obtained by convolutional coding and pooling to compensate for displacement information. Describes the norm 1. Represents the L2 norm, Indicates cascading. Indicates the first weighted hyperparameter. The displacement information compensation factor is represented. This represents the length of the semantic feature vector of the customer complaint text content. This represents the second weighted hyperparameter. This represents the feature vector for classifying customer complaint issues.

[0057] In the technical solution of this application, during the fusion of the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint, the fine-grained dimensional offset between the two feature vectors may lead to local structural collapse in the fused customer complaint problem classification feature vector. Therefore, in the technical solution of this application, backpropagation displacement information compensation based on semantic space amplitude is performed on the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint.

[0058] Specifically, based on the limited representation of the foreground-background structure distinction of the high-dimensional manifold of the customer complaint text content semantic feature vector and the customer complaint multimodal association feature vector, a fine-grained correspondence of key features between the customer complaint text content semantic feature vector and the customer complaint multimodal association feature vector is modeled. Then, backpropagation displacement compensation information between the corresponding feature values ​​of the customer complaint text content semantic feature vector and the customer complaint multimodal association feature vector is used to actively adjust the fine-grained correspondence. This is done by focusing on avoiding the imbalance at the feature vector level of the customer complaint text content semantic feature vector and the customer complaint multimodal association feature vector, thereby synchronizing the micro-dimension of the two feature vectors. This ensures that the information between the feature vectors can be effectively aligned and combined during the fusion process, thus avoiding the abnormal accumulation of local structures.

[0059] Furthermore, the customer complaint issue classification feature vectors are obtained by fusing semantic features of the text content with correlation features from other modalities. These vectors contain the core information and context of the complaint. These feature vectors are then input into a classifier, which uses model algorithms to transform these high-dimensional features into specific category labels. By passing the feature vectors to the classifier, businesses can automate the processing of large amounts of complaint data. Manually processing each complaint is not only time-consuming but also prone to errors, while the classifier can quickly and consistently categorize complaints. This automated processing method significantly improves efficiency, ensuring that all complaints are processed and responded to promptly and accurately.

[0060] In summary, this application first acquires customer complaint text data, customer complaint audio data, and customer complaint image data collected by the complaint platform. Then, it uses deep learning technology to extract features and perform correlation analysis on the three data. Finally, it uses a classifier to obtain customer complaint problem classification labels, thereby achieving more comprehensive and accurate problem identification. This helps enterprises respond quickly, improve products or services, optimize complaint management processes, and enhance customer satisfaction and loyalty.

[0061] Figure 5 This is a block diagram of an intelligent customer complaint management device according to an embodiment of this application. Figure 5 As shown, the intelligent customer complaint management device 100 according to an embodiment of this application includes: a complaint platform data acquisition module 110, used to acquire customer complaint text data, customer complaint audio data, and customer complaint image data collected by the complaint platform; a complaint platform data extraction module 120, used to extract customer complaint text content semantic feature vectors and customer complaint multimodal association feature vectors from the customer complaint text data, the customer complaint audio data, and the customer complaint image data collected by the complaint platform; and a customer complaint problem label classification module 130, used to obtain customer complaint problem classification labels based on the customer complaint text content semantic feature vectors and the customer complaint multimodal association feature vectors.

[0062] Here, those skilled in the art will understand that the specific operations of each step in the aforementioned intelligent customer complaint management device have been referenced above. Figures 1 to 4 The description of the intelligent management method for customer complaints is detailed here, and therefore, its repeated description will be omitted.

[0063] As described above, the intelligent customer complaint management device 100 according to the embodiments of this application can be implemented in various terminal devices. In one example, the intelligent customer complaint management device 100 can be integrated into the terminal device as a software module and / or a hardware module. For example, the intelligent customer complaint management device 100 can be a software module in the operating system of the terminal device, or it can be an application developed for the terminal device; of course, the intelligent customer complaint management device 100 can also be one of many hardware modules of the terminal device.

[0064] Alternatively, in another example, the intelligent customer complaint management device 100 and the terminal device can also be separate devices, and the intelligent customer complaint management device 100 can be connected to the terminal device via wired and / or wireless networks, and transmit interactive information in accordance with an agreed data format.

[0065] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0066] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0067] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0068] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0069] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0070] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit of the technical solutions of the present invention.

Claims

1. A method for intelligent management of customer complaints, characterized in that, include: Acquire customer complaint text data, customer complaint audio data, and customer complaint image data collected by the complaint platform; Extracting semantic feature vectors of customer complaint text content and multimodal association feature vectors of customer complaints from the customer complaint text data, customer complaint audio data and customer complaint image data collected by the complaint platform; including: performing feature extraction on the customer complaint audio data to obtain a global feature vector of the customer complaint speech spectrum; Feature extraction is performed on the customer complaint image data to obtain the customer complaint image information feature vector; The global feature vector of the customer complaint speech spectrum and the feature vector of the customer complaint image information are correlated to obtain the multimodal correlation feature vector of the customer complaint; Based on the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint, a customer complaint issue classification label is obtained, including: Calculate the positional difference between the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint to obtain the displacement information compensation difference feature vector; The displacement information compensation dot product feature vector is obtained by calculating the position-based dot product between the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint. The displacement information compensation feature vector is obtained by calculating the positional summation between the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint. The displacement information compensation differential feature vector, the displacement information compensation position-based dot product feature vector, and the displacement information compensation position-based summation feature vector are concatenated and input into a one-dimensional convolutional layer and then subjected to max pooling to obtain the displacement information compensation convolutional coding pooled feature vector. The square root of the L2 norm of the displacement information compensation convolutional coding pooling feature vector is weighted and added to the L1 norm of the displacement information compensation convolutional coding pooling feature vector to obtain the displacement information compensation factor. The displacement information compensation intermediate vector is obtained by adding the displacement information compensation feature vector by position to the root value of the length of the semantic feature vector of the customer complaint text content by position. Using the displacement information compensation factor as the first weighting coefficient and the second weighting hyperparameter as the second weighting coefficient, the position-weighted sum of the displacement information compensation feature vector and the displacement information compensation intermediate vector is calculated to obtain the customer complaint problem classification feature vector.

2. The intelligent customer complaint management method according to claim 1, characterized in that, Extracting semantic feature vectors of customer complaint text content and multimodal association feature vectors of customer complaints from the customer complaint text data, customer complaint audio data, and customer complaint image data collected by the complaint platform, further includes: Feature extraction is performed on the customer complaint text data to obtain the semantic feature vector of the customer complaint text content.

3. The intelligent customer complaint management method according to claim 2, characterized in that, Feature extraction is performed on the customer complaint text data to obtain the semantic feature vector of the customer complaint text content, including: The customer complaint text data is preprocessed to obtain a sequence of word embedding vectors for the customer complaint text content; The word embedding vector sequence of the customer complaint text content is processed by a converter-based customer complaint text content semantic encoder model to obtain the semantic feature vector of the customer complaint text content.

4. The intelligent customer complaint management method according to claim 3, characterized in that, Feature extraction is performed on the customer complaint audio data to obtain a global feature vector of the customer complaint speech spectrum, including: The customer complaint audio data is preprocessed to obtain a multi-channel speech spectrogram of the customer complaint. The customer complaint multi-channel speech spectrogram is processed through a customer complaint multi-channel speech spectrogram convolutional neural network based on a two-stream network model to obtain the global feature vector of the customer complaint speech spectrogram.

5. The intelligent customer complaint management method according to claim 4, characterized in that, The customer complaint audio data is preprocessed to obtain a multi-channel speech spectrogram of the customer complaint, including: The customer complaint audio data is processed by a customer complaint audio information noise reduction module based on an automatic encoder to obtain noise-reduced customer complaint audio information. Extract the logarithmic Melp plot, cochlear spectrogram, and constant Q transform spectrogram of the customer complaint audio from the noise-reduced customer complaint audio information. The logarithmic Melp plot, cochlear spectrogram, and constant Q transform spectrogram of the customer complaint audio are arranged to form the multi-channel speech spectrogram of the customer complaint.

6. The intelligent customer complaint management method according to claim 5, characterized in that, Feature extraction is performed on the customer complaint image data to obtain a customer complaint image information feature vector, including: The customer complaint image data is processed by a customer complaint image information feature encoder based on a deep neural network to obtain a customer complaint image information feature map. The feature map of the customer complaint image information is pooled to obtain the feature vector of the customer complaint image information.

7. The intelligent customer complaint management method according to claim 6, characterized in that, Based on the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint, a customer complaint issue classification label is obtained, including: The semantic feature vector of customer complaint text content and the multimodal association feature vector of customer complaint are compensated by backpropagation displacement information based on semantic space amplitude to obtain the customer complaint problem classification feature vector; The customer complaint issue classification feature vector is passed through a classifier to obtain a classification result, which is used to represent the customer complaint issue classification label.

8. A customer complaint intelligent management device, implementing the customer complaint intelligent management method as described in any one of claims 1-7, characterized in that, include: The complaint platform data acquisition module is used to acquire customer complaint text data, customer complaint audio data, and customer complaint image data collected by the complaint platform. The complaint platform data extraction module is used to extract semantic feature vectors of customer complaint text content and multimodal association feature vectors of customer complaints from the customer complaint text data, customer complaint audio data and customer complaint image data collected by the complaint platform; The customer complaint issue label classification module is used to obtain customer complaint issue classification labels based on the semantic feature vector of the customer complaint text content and the multimodal association feature vector of the customer complaint.

9. An electronic device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the intelligent customer complaint management method as described in any one of claims 1 to 7.

Images