An intelligent e-commerce customer service after-sales processing method, system, device and medium

Abstract

The application provides an intelligent e-commerce customer service after-sales processing method, system, device and medium, and relates to the technical field of intelligent customer service. The method comprises the following steps: obtaining text data, image data, conversation history data and compensation mechanism associated with after-sales problems; performing feature processing on the text data and the image data to obtain text features and image features; obtaining the importance ratio between the text features and the image features according to the conversation history data; combining the importance ratio, performing interactive feature enhancement processing and fusion processing on the text features and the image features to obtain problem classification results and problem classification confidence; generating processing information corresponding to the after-sales problems according to the problem classification results, the problem classification confidence and the compensation mechanism; and processing the after-sales problems according to the problem classification results, the problem classification confidence and the processing information corresponding to the after-sales problems. The application improves the accuracy of after-sales problem classification by comprehensively analyzing the text and the image, thereby optimizing the consumer experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent customer service, and in particular to an intelligent e-commerce customer service after-sales processing method, system, device and medium. BACKGROUND

[0002] With the rapid development of e-commerce, consumers' demand for after-sales service is increasing. The existing intelligent customer service after-sales processing technology mainly relies on text processing and simple question and answer mode, which is difficult to cope with complex after-sales problems. Especially when consumers upload pictures to show product damage, stains or missing parts, the existing technology often cannot accurately analyze the picture content and cannot provide effective solutions. In addition, most of the existing intelligent customer service after-sales processing technology cannot effectively integrate picture and text information, resulting in low problem identification accuracy, increasing the processing complexity and time consumption, and reducing the consumer experience. SUMMARY

[0003] The present application provides an intelligent e-commerce customer service after-sales processing method, system, device and medium to solve one or more technical problems existing in the prior art, at least to provide beneficial options or create conditions.

[0004] In one aspect, the present application provides an intelligent e-commerce customer service after-sales processing method, comprising the following steps:

[0005] Obtaining text data, image data, dialogue history data and compensation mechanism associated with after-sales problems;

[0006] Performing feature processing on the text data and the image data to obtain text features and image features;

[0007] According to the dialogue history data, obtaining the importance ratio between the text features and the image features;

[0008] Combining the importance ratio, performing interactive feature enhancement processing and fusion processing on the text features and the image features to obtain problem classification results and problem classification confidence;

[0009] According to the problem classification results, the problem classification confidence and the compensation mechanism, generating processing information corresponding to the after-sales problems;

[0010] According to the problem classification results, the problem classification confidence and the processing information corresponding to the after-sales problems, processing the after-sales problems.

[0011] Further, the feature processing on the text data and the image data to obtain text features and image features comprises:

[0012] perform feature processing on the text data by using a preset text processing model to obtain the text feature;

[0013] perform feature processing on the image data by using a preset image processing model to obtain the image feature.

[0014] Further, the obtaining of the importance ratio between the text feature and the image feature according to the dialogue history data comprises:

[0015] perform feature processing on the dialogue history data by using a preset text processing model to obtain a dialogue history vector;

[0016] obtain the importance ratio between the text feature and the image feature according to the dialogue history vector.

[0017] Further, the interactive feature enhancement processing and fusion processing of the text feature and the image feature in combination with the importance ratio to obtain the question classification result and the question classification confidence level comprises:

[0018] perform linear transformation on the text feature and the image feature in combination with the importance ratio to obtain weighted text features and weighted image features of the same dimension;

[0019] perform interactive feature enhancement processing and fusion processing on the weighted text features and the weighted image features by using a deep learning model to obtain the question classification result and the question classification confidence level.

[0020] Further, the interactive feature enhancement processing and fusion processing of the weighted text features and the weighted image features by using a deep learning model to obtain the question classification result and the question classification confidence level comprises:

[0021] obtain a self-enhanced text feature by using a self-attention mechanism of the deep learning model according to the weighted text features;

[0022] obtain a self-enhanced image feature by using a self-attention mechanism of the deep learning model according to the weighted image features;

[0023] obtain cross-modal enhanced text features and cross-modal enhanced image features by using a cross-modal attention mechanism of the deep learning model according to the weighted text features and the weighted image features;

[0024] obtain the question classification result and the question classification confidence level by using a multi-layer perception model according to the self-enhanced text features, the self-enhanced image features, the cross-modal enhanced text features and the cross-modal enhanced image features.

[0025] Further, the processing of the after-sales problem according to the problem classification result corresponding to the after-sales problem, the problem classification confidence and the processing information comprises:

[0026] If the problem classification confidence corresponding to the after-sales problem exceeds a preset confidence threshold, the after-sales problem is automatically processed according to the problem classification result corresponding to the after-sales problem and the processing information;

[0027] Otherwise, the problem classification result corresponding to the after-sales problem and the processing information are sent to an artificial customer service for processing the after-sales problem by the artificial customer service.

[0028] In another aspect, the present application provides an intelligent e-commerce customer service after-sales processing system, comprising a data acquisition module, a feature processing module, a weight adjustment module, a feature interaction module and an after-sales processing module;

[0029] The data acquisition module is configured to acquire text data, image data, dialogue history data and a compensation mechanism associated with an after-sales problem;

[0030] The feature processing module is configured to perform feature processing on the text data and the image data to obtain text features and image features;

[0031] The weight adjustment module is configured to obtain an importance ratio between the text features and the image features according to the dialogue history data;

[0032] The feature interaction module is configured to perform interactive feature enhancement processing and fusion processing on the text features and the image features in combination with the importance ratio to obtain a problem classification result and a problem classification confidence;

[0033] The after-sales processing module is configured to generate processing information corresponding to the after-sales problem according to the problem classification result, the problem classification confidence and the compensation mechanism, and to process the after-sales problem according to the problem classification result corresponding to the after-sales problem, the problem classification confidence and the processing information.

[0034] Further, the after-sales processing module comprises an automatic processing unit and an artificial processing unit;

[0035] The automatic processing unit is configured to automatically process the after-sales problem according to the problem classification result corresponding to the after-sales problem and the processing information when the problem classification confidence corresponding to the after-sales problem exceeds a preset confidence threshold;

[0036] The artificial processing unit is configured to send the problem classification result and the processing information corresponding to the after-sales problem to an artificial customer service when a problem classification confidence corresponding to the after-sales problem does not exceed a preset confidence threshold, so as to facilitate the artificial customer service to process the after-sales problem.

[0037] In another aspect, the present application provides an intelligent e-commerce customer service after-sales processing device, comprising a processor and a memory, the memory is used for storing a program; when the program is executed by the processor, the processor implements the foregoing intelligent e-commerce customer service after-sales processing method.

[0038] In another aspect, the present application provides a computer readable storage medium, wherein a processor executable program is stored, the processor executable program is used for implementing the foregoing intelligent e-commerce customer service after-sales processing method when executed by a processor.

[0039] The present application has the following beneficial effects: the present application provides an intelligent e-commerce customer service after-sales processing method, comprising: obtaining text data, image data, dialogue history data and compensation mechanism associated with an after-sales problem; performing feature processing on the text data and the image data to obtain text features and image features; obtaining the importance ratio between the text features and the image features according to the dialogue history data; combining the importance ratio, the text features and the image features are processed by interactive feature enhancement processing and fusion processing to obtain problem classification result and problem classification confidence; generating processing information corresponding to the after-sales problem according to the problem classification result, the problem classification confidence and the compensation mechanism; processing the after-sales problem according to the problem classification result, the problem classification confidence and the processing information corresponding to the after-sales problem. The present application improves the accuracy of after-sales problem classification by comprehensively analyzing text, image and dialogue history data, thereby optimizing the consumer experience. The present application also provides corresponding devices, systems and media, and the beneficial effects of the devices, systems and media are similar to those of the method, which will not be described here.

[0040] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application can be achieved and obtained by the structures particularly pointed out in the specification, claims and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings are used to provide a further understanding of the technical solutions of the present application, and constitute a part of the specification, and are used to explain the technical solutions of the present application together with embodiments of the present application, and do not constitute a limitation on the technical solutions of the present application.

[0042] Figure 1 is a flowchart of an intelligent e-commerce customer service after-sales processing method provided by the present application;

[0043] Figure 2 is a schematic diagram of determining a question classification result and a question classification confidence provided by the present application;

[0044] Figure 3 is a structural diagram of an intelligent e-commerce customer service after-sales processing system provided by the present application;

[0045] Figure 4 is a structural diagram of an intelligent e-commerce customer service after-sales processing device provided by the present application. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0047] The present application is further described below in combination with the drawings and specific embodiments of the present application. The described embodiments should not be considered as limiting the present application, and all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0048] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0050] The rapid development of e-commerce has increased the demand for after-sales service, and has also improved the requirements of consumers on shopping experience. The traditional manual customer service is inefficient in handling after-sales problems, and it is difficult to respond quickly to repetitive inquiries, consumes a lot of time and effort, and is easy to cause customer dissatisfaction due to errors or omissions, especially when dealing with routine requests such as refunds and exchanges, the process is cumbersome. Although the existing intelligent customer service system has reduced the workload to a certain extent, the stereotyped response based on the question and answer mode is of limited help for complex problems, and lacks the ability to integrate and analyze image and text information. When consumers upload pictures of product problems and attach descriptions, most existing intelligent customer service systems cannot effectively associate the two types of information, resulting in an inability to accurately understand customer needs, increasing the complexity and time cost of communication, and resulting in a poor consumer experience.

[0051] To address the problems and defects of the prior art, the embodiments of the present application provide an intelligent e-commerce customer service after-sales processing method, system, device and medium. The embodiments of the present application comprehensively analyze the text and image features by obtaining text data, image data, dialogue history data and compensation mechanisms associated with after-sales problems, and combine the importance proportion of the dialogue history for interactive feature enhancement and fusion processing, thereby improving the accuracy and confidence of problem classification. For high-confidence problems, the system automatically processes and generates detailed processing information to send to the customer; and for low-confidence problems, the system sends preliminary processing suggestions to the artificial customer service for further review. This method not only improves the accuracy of problem identification, enhances the processing efficiency, optimizes the user experience, but also reduces the operating cost, effectively solves the shortcomings of the existing intelligent customer service system in multi-modal information integration analysis and complex problem processing, thereby improving the quality and efficiency of e-commerce after-sales service.

[0052] First, the implementation steps of the intelligent e-commerce customer service after-sales processing method provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0053] The intelligent e-commerce customer service after-sales processing method proposed in the embodiments of the present application can be applied in a terminal, can also be applied in a server, and can also be software running in a terminal or a server, etc. The terminal can be a tablet computer, a notebook computer, a desktop computer, etc., but is not limited thereto. The server can be a standalone physical server, can also be a server cluster or a distributed system composed of multiple physical servers, and can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks, and basic cloud computing services such as big data and artificial intelligence platforms.

[0054] Referring to Figure 1 The implementation process of the intelligent e-commerce customer service after-sales processing method provided by the embodiments of the present application can include but is not limited to the following steps.

[0055] Step 101, obtaining text data, image data, dialogue history data and compensation mechanisms associated with after-sales problems.

[0056] In step 101, all necessary information related to after-sales problems is collected. By obtaining text data (such as customer description), image data (such as product problem photos), dialogue history data (such as previous interaction records), and compensation mechanisms (such as company policies), the system can obtain a complete context view, ensuring that subsequent processing is based on a sufficient information base. This not only helps to improve the accuracy of problem classification, but also provides a basis for generating reasonable processing information, thereby improving the overall service quality.

[0057] Step 102, feature processing is performed on the text data and image data to obtain text features and image features.

[0058] In step 102, feature processing is performed on the text data and image data to obtain text features and image features. The text features can capture the customer's description intent and after-sales demand, while the image features can reveal the specific condition of the goods (such as damage degree, stain position, etc.). By processing the features of these two modalities, the system can obtain more structured and interpretable information representation, laying a solid foundation for subsequent feature fusion and problem classification.

[0059] Step 103, according to the dialogue history data, the importance ratio between the text features and the image features is obtained.

[0060] In step 103, by analyzing the dialogue history data, the relative importance ratio between the text features and the image features is calculated. The dialogue history data provides rich contextual information, which can help the system understand which information is more critical to the current problem. For example, in some cases, images may be more intuitive than text descriptions to reflect the nature of the problem; while in other cases, detailed text descriptions may be the key to solving the problem. Determining the importance ratio helps to more accurately weigh the influence of different modal information in subsequent steps, improving the accuracy of problem classification.

[0061] Step 104, interactive feature enhancement processing and fusion processing are performed on the text features and image features in combination with the importance ratio to obtain problem classification results and problem classification confidence.

[0062] In step 104, interactive feature enhancement and fusion processing are performed on the text features and image features in combination with the importance ratio obtained in the previous step. This processing method not only strengthens the feature representation of each modality, but also establishes the correlation between the two, forming a more comprehensive and comprehensive problem representation. Finally, the system outputs the problem classification results and their confidence scores, ensuring that subsequent processing decisions are based on reliable data support, improving the accuracy and credibility of automated processing.

[0063] Step 105, according to the problem classification results, problem classification confidence and compensation mechanism, the processing information corresponding to the after-sales problem is generated.

[0064] In step 105, based on the problem classification result and the confidence score, combined with the company's compensation mechanism, detailed after-sales processing information is automatically generated. This includes specific processing information (such as refund, exchange, repair, etc.), operation guidelines (such as return process, preparation of goods packaging, etc.), time frame (such as estimated processing time), and other additional information (such as ways to contact customer service). The generated processing information not only conforms to the company's policy, but also meets the needs of customers, ensuring that the processing process is transparent and efficient, and improving customer trust and satisfaction.

[0065] In step 106, according to the problem classification result, problem classification confidence and processing information corresponding to the after-sales problem, the after-sales problem is processed.

[0066] In step 106, according to the problem classification result, problem classification confidence and processing information corresponding to the after-sales problem, the generated processing information is applied to the actual problem processing. This method not only improves the processing efficiency, but also reduces the customer waiting time, thereby improving the quality of after-sales service and customer experience.

[0067] Through the above steps, the intelligent e-commerce customer service after-sales processing method provided by the embodiments of the present application realizes the full-process automation and intelligentization from data acquisition to problem solving, effectively improves the accuracy and efficiency of after-sales service, optimizes user experience, and reduces operating costs.

[0068] In some embodiments of the present application, with reference to Figure 2 In step 102, the implementation process of the feature processing of the text data and the image data to obtain the text features and the image features can include but is not limited to the following steps.

[0069] In step 201, the text data is processed by using a preset text processing model to obtain text features.

[0070] In step 201, the text data provided by the customer is processed by using a preset text processing model to extract features. Through deep learning technology, the semantic information and context relationship in the text can be captured to generate high-dimensional text feature vectors. The text features not only contain the information at the word level, but also reflect the overall semantic structure of the sentence or paragraph, so that the system can more accurately understand the customer's problem description and provide strong support for subsequent problem classification.

[0071] In some embodiments of the present application, the text processing model includes a RoBERTa model.

[0072] The RoBERTa model is an improved version of the BERT model that effectively enhances the performance and robustness of the model through a series of optimization measures, such as a larger training dataset, a dynamic masking strategy, and so on. The RoBERTa model, through large-scale pre-training, can capture deep semantic information and contextual relationships in text. When handling after-sales problems, the system uses the RoBERTa model to extract features from the text data provided by the customer, generating high-dimensional text feature vectors. These features not only contain lexical-level information but also reflect the overall semantic structure of sentences or paragraphs, allowing the system to more accurately understand the customer's description intent and emotional tendency, thereby improving the accuracy of problem classification.

[0073] Moreover, the RoBERTa model has strong natural language processing capabilities, especially in handling complex and diverse text data. By using the text features extracted by the RoBERTa model, the system can more accurately classify problems in subsequent steps and generate reliable problem classification confidence scores. This high-precision classification result helps the automated processing module quickly respond to simple problems while ensuring that complex problems can be correctly identified and transferred to human customer service for review, improving the reliability and efficiency of overall processing.

[0074] In some embodiments of the present application, the implementation process of the text feature processing using the preset RoBERTa model can include but is not limited to the following steps.

[0075] First, the input text data is subjected to necessary preprocessing operations such as word segmentation, removal of stop words, conversion to lowercase, etc., to ensure that the text sequence input into the RoBERTa model is clean and formatted. For example, the Jieba word segmentation tool is used to segment the text into words or subwords.

[0076] Second, the RoBERTa model receives the preprocessed text sequence as input. Each word or subword (token) is mapped to an embedding vector. Assuming that the text sequence contains tokens, the input embedding can be represented as where represents the embedding vector of the token, and the embedding dimension is These embedding vectors integrate lexical, positional, and segment information, allowing the model to understand the context of the text.

[0077] Furthermore, the RoBERTa model processes the input embedding through multiple layers of Transformer encoders. Each layer of the encoder includes a self-attention mechanism to capture the relationships between different parts of the text. The self-attention mechanism satisfies the following equation (1):

[0078] (1);

[0079] In formula (1), Represents the query matrix. ; Represents the key matrix, ; Represents a value matrix, ; , and It is a learnable weight matrix. It is a key matrix The dimension is used to scale the dot product to stabilize the gradient. Through the self-attention mechanism, the model is able to capture long dependencies and contextual information in the text.

[0080] Then, each Transformer encoder layer in the RoBERTa model includes a layer normalization function and a feedforward neural network layer. The layer normalization function helps accelerate training and improve model performance, while the feedforward neural network layer further enhances the feature representation capability. After multiple encoder layers, the final output hidden state is represented as follows: ,in express The corresponding hidden states contain rich semantic information.

[0081] Finally, the text features are aggregated and output. To obtain the feature representation of the entire text, the system typically performs some form of aggregation on the hidden states of all tokens. Common methods include taking the hidden state of the last token (i.e., the [CLS] tag), mean pooling, or max pooling. Assuming mean pooling is chosen, the text features... The following formula (2) must be satisfied:

[0082] (2).

[0083] Through the above steps, this embodiment of the application utilizes a pre-defined RoBERTa model to perform feature processing on text data, generating text features that accurately capture text semantics and contextual relationships, providing a solid foundation for subsequent interactive feature enhancement and fusion processing. This method not only improves the accuracy of problem classification but also enhances the system's understanding and processing capabilities.

[0084] Step 202: Use a preset image processing model to perform feature processing on the image data to obtain image features.

[0085] In step 202, a pre-set image processing model is used to extract features from the product picture uploaded by the consumer. These models can automatically learn key visual features in the image, such as shape, color, texture, etc., and convert them into low-dimensional feature vectors. Image features can help the system intuitively identify the state of the product (such as damage, stains, missing parts, etc.), and combined with text features, form a more comprehensive representation of the problem. In this way, the system can make more accurate judgments based on multi-modal data, effectively improving the accuracy and reliability of problem classification.

[0086] In some embodiments of the present application, the image processing model includes a ResNet model, and a Mask R-CNN model is combined based on the ResNet model. The ResNet model ensures that the system can accurately capture key information in the product picture through its powerful image feature extraction capability; and the Mask R-CNN model further enhances the accuracy of object detection and segmentation, enabling the system to more carefully understand the specific circumstances of the problem.

[0087] Specifically, the ResNet model solves the problem of gradient disappearance in deep neural networks by introducing residual connections, allowing the network to be deeper and more effective in training. The ResNet model has strong image feature extraction capability, capable of capturing low-level features (such as edges, textures) and high-level semantic information (such as object shape, color distribution) in images. In embodiments of the present application, the ResNet model is used to perform preliminary feature extraction on the uploaded product picture, generating a high-dimensional image feature vector.

[0088] In addition, the Mask R-CNN model is an improvement based on the Faster R-CNN model, which not only can perform object detection (identify objects and their positions in the image), but also can generate pixel-level segmentation masks for each detected object. This capability is particularly important for handling complex after-sales problems, for example, when a customer uploads a picture of a damaged product, the Mask R-CNN model can help the system accurately locate and segment the damaged area, thereby better understanding the specific circumstances of the problem.

[0089] In some embodiments of the present application, the ResNet model combined with the Mask R-CNN model is used to process image data, and the implementation process of image feature extraction can include but is not limited to the following steps.

[0090] First, the image data is pre-processed to obtain an input image. Specifically, the image data is resized, cropped, normalized, etc., to obtain an input image that meets the model input requirements.

[0091] Secondly, feature extraction is performed on the input image. A ResNet model is used to extract high-level semantic features from the input image. The ResNet model solves the gradient vanishing problem in deep neural networks by introducing residual blocks and allows the construction of deeper network structures. After the input image passes through the ResNet model, a feature map rich in semantic information is output.

[0092] Furthermore, a region proposal network in the Mask R-CNN model is used to generate a series of candidate regions based on the feature map generated by ResNet. These candidate regions are the basis for subsequent processing and define the positions that may contain objects of interest. In order to obtain a fixed-size feature map from each candidate region, the Mask R-CNN model performs candidate region alignment, preserving more spatial information.

[0093] Then, for each candidate region, the Mask R-CNN model performs three tasks in parallel: predicting the position of the bounding box, classifying the category, and predicting the mask. The classification task is used to determine the category of the object within the region, the bounding box position prediction task is used to fine-tune the position of the candidate region to more accurately match the position of the real object, and the mask prediction task is used to generate a binary mask matching the size of the candidate region to represent the precise contour of the object.

[0094] Finally, the classification results, bounding boxes, and masks output by the Mask R-CNN model are combined to obtain the precise positioning and classification information of each instance in the image as image features.

[0095] In some embodiments of the present application, with reference to Figure 2 In step 103, the implementation process of obtaining the importance ratio between the text features and the image features based on the dialogue history data can include but is not limited to the following steps.

[0096] Step 301: Use a pre-set text processing model to perform feature processing on the dialogue history data to obtain a dialogue history vector.

[0097] In step 301, the dialogue history vector is generated by using a pre-set text processing model to extract features from the previous dialogue history data. These models can capture semantic information, sentiment orientation, and contextual relationships in the dialogue, generating high-dimensional dialogue history feature vectors. The dialogue history vector not only contains lexical level information, but also reflects the interaction patterns and key topics between the customer and the customer service during the entire dialogue process. This helps the system better understand the background of the current problem and provides a basis for subsequent determination of the importance ratio between the text and image features.

[0098] Step 302: Obtain the importance ratio between the text features and the image features based on the dialogue history vector.

[0099] In step 302, based on the generated dialogue history vector, the relative importance of the text features and the image features in the current question is evaluated. Specifically, the importance parameter corresponding to the text features and the importance parameter corresponding to the image features are calculated respectively, and then the importance ratio between the text features and the image features is obtained. By analyzing the keywords, high-frequency topics and customer concerns in the dialogue history, the system can identify which information is more critical to solving the problem. For example, if the customer mentions the specific damage of the goods multiple times and uploads related pictures, the image features may be more important than the text description; on the contrary, if the customer describes the service experience or process problem in detail, the text features may be more important. In this way, the system can dynamically adjust the weights of the text and image features to ensure that the most relevant information is fully utilized in subsequent processing, improving the accuracy of problem classification and the effectiveness of processing information.

[0100] In some embodiments of the present application, the importance parameter corresponding to the text features satisfies the following formula (3):

[0101] (3);

[0102] In formula (3), represents the importance parameter corresponding to the text features, represents the dialogue history vector, represents the random weight matrix corresponding to the text features, represents the random weight matrix corresponding to the image features; the importance parameter corresponding to the image features is represented as: .

[0103] In some embodiments of the present application, referring to Figure 2 , the implementation process of combining the importance ratio to perform interactive feature enhancement processing and fusion processing on the text features and the image features to obtain the problem classification result and the problem classification confidence in step 104 can include but is not limited to the following steps.

[0104] Step 401, combine the importance ratio to perform linear transformation on the text features and the image features to obtain weighted text features and weighted image features of the same dimension.

[0105] In step 401, according to the previously calculated relative importance ratio between the text features and the image features, linear transformation is performed on the two kinds of features to make them have the same dimension. Specifically, the system multiplies the text features and the image features by the corresponding weight coefficients respectively to obtain the weighted text features and the weighted image features. This linear transformation ensures that the data of different modalities can be effectively compared and fused in the same space, providing a unified basis for subsequent interactive feature enhancement processing. In this way, the system can more accurately capture the importance of each modality information, thereby improving the accuracy of the overall feature representation.

[0106] In step 402, the weighted text features and the weighted image features are subjected to interactive feature enhancement processing and fusion processing using a deep learning model to obtain the problem classification result and the problem classification confidence.

[0107] In step 402, the weighted text features and the weighted image features are subjected to interactive feature enhancement processing and fusion processing using a deep learning model to obtain the problem classification result and the problem classification confidence.

[0108] Through the above steps, the embodiments of the present application realize efficient interactive enhancement and fusion processing of text features and image features, ensuring that the system can comprehensively understand the after-sales problems from different angles and generate accurate problem classification results and confidence scores. This not only enhances the decision-making ability of the system, but also provides a solid foundation for subsequent automated processing or manual review, ensuring that customer problems are solved in a timely and effective manner.

[0109] In some embodiments of the present application, with reference to Figure 2 In step 402, the weighted text features and the weighted image features are subjected to interactive feature enhancement processing and fusion processing using a deep learning model to obtain the problem classification result and the problem classification confidence.

[0110] In step 501, according to the weighted text features, the self-attention mechanism of the deep learning model is used to obtain self-enhanced text features.

[0111] In step 501, the weighted text features are processed using the self-attention mechanism in the deep learning model. The self-attention mechanism can capture the relationships between different parts of the text, emphasizing those words or sentence fragments that are more important for the current problem. In this way, the system can generate more rich and representative self-enhanced text features. These features not only contain the original text information, but also strengthen the understanding of key content, thereby improving the accuracy of problem classification.

[0112] In step 502, based on the weighted image features, the self-attention mechanism of the deep learning model is used to obtain self-enhanced image features.

[0113] In step 502, the weighted image features are also processed using the self-attention mechanism of the deep learning model. The self-attention mechanism can capture the relationships between different regions within the image, emphasizing those visual elements (such as damaged parts, stain locations, etc.) that are more important for the current problem. In this way, the system can generate more detailed and representative self-enhanced image features. These features not only retain the key visual information of the original image, but also enhance the understanding of the problem-related areas, further improving the accuracy of problem classification.

[0114] In step 503, based on the weighted text features and the weighted image features, the cross-modal attention mechanism of the deep learning model is used to obtain cross-modal enhanced text features and cross-modal enhanced image features.

[0115] In step 503, the cross-modal attention mechanism of the deep learning model is used to combine the weighted text features and the weighted image features. The cross-modal attention mechanism can establish a connection between the two modalities, enabling the system to understand the mutual influence between text and image. For example, if the text describes the damage of a certain area, and the image shows the details of that area, the cross-modal attention mechanism can help the system better integrate these two pieces of information. In this way, the cross-modal enhanced text features and cross-modal enhanced image features generated by the system not only more comprehensively reflect the overall picture of the problem, but also improve the accuracy and confidence of problem classification.

[0116] In step 504, based on the self-enhanced text features, the self-enhanced image features, the cross-modal enhanced text features, and the cross-modal enhanced image features, a multi-layer perceptron model is used to obtain the problem classification result and the problem classification confidence.

[0117] In step 504, the four enhanced features (self-enhanced text feature, self-enhanced image feature, cross-modal enhanced text feature, and cross-modal enhanced image feature) are input into a multi-layer perception model for final question classification. The multi-layer perception model can effectively extract information that best represents the question from a high-dimensional feature space through multiple layers of nonlinear transformation and output the question classification result and its corresponding confidence score. This method not only improves the accuracy of question classification, but also provides a reliable confidence evaluation to help the system decide whether manual review is needed. In this way, the system can make more accurate judgments based on multi-modal data, effectively improving the quality and efficiency of after-sales service.

[0118] In some embodiments of the present application, the deep learning model includes a Transformer model.

[0119] In some embodiments of the present application, the implementation process of using the Transformer model to perform interactive feature enhancement processing and fusion processing on the weighted text feature and the weighted image feature to obtain the question classification result and the question classification confidence can include but is not limited to the following steps.

[0120] First, assume that the weighted text feature is and the weighted image feature is , and their dimensions are . According to the weighted text feature , the self-enhanced text feature is obtained using the self-attention mechanism of the Transformer model. According to the weighted image feature , the self-enhanced image feature is obtained using the self-attention mechanism of the Transformer model. The core formula of the self-attention mechanism of the Transformer model is referred to as formula (1).

[0121] Second, according to the weighted text feature and the weighted image feature , the cross-modal enhanced text feature and the cross-modal enhanced image feature are obtained using the cross-modal attention mechanism of the Transformer model. The cross-modal enhanced text feature satisfies the following formula (4):

[0122] (4);

[0123] In formula (4), represents the learning rate parameter of , which controls the update amplitude of , denotes an attention score of the weighted image feature based on the weighted text feature, ; cross-modal enhanced image feature satisfies the following formula (5):

[0124] (5);

[0125] In formula (5), denotes a learning rate parameter of controls the update amplitude of denotes an attention score of the weighted image feature based on the weighted text feature, .

[0126] Then, according to the self-enhanced text feature , the self-enhanced image feature , the cross-modal enhanced text feature and the cross-modal enhanced image feature , interactive iterative updates are performed to obtain the final enhanced text feature and , and the problem classification probability distribution and are obtained by using a multilayer perceptron model. satisfies the following formula (6):

[0127] (6);

[0128] In formula (6), denotes a multilayer perceptron model, denotes a learnable weight parameter related to , and denotes a learnable bias parameter related to .

[0129] Finally, according to the problem classification probability distribution , a problem classification result and a problem classification confidence are obtained.

[0130] Through the above steps, the embodiments of the present application realize efficient interactive enhancement and fusion processing of the weighted text feature and the weighted image feature, ensuring that the system can comprehensively understand the after-sales problem from different angles, and generate accurate problem classification results and confidence scores. This not only enhances the decision-making ability of the system, but also provides a solid foundation for subsequent automated processing or manual review, ensuring that customer problems are solved in a timely and effective manner.

[0131] ​In some embodiments of the present application, in step 106, the implementation process of handling the after-sales problem according to the problem classification result corresponding to the after-sales problem, the problem classification confidence and the processing information can include but is not limited to the following steps.

[0132] If the problem classification confidence corresponding to the after-sales problem exceeds the preset confidence threshold, the after-sales problem is automatically handled according to the problem classification result corresponding to the after-sales problem and the processing information. Otherwise, the problem classification result corresponding to the after-sales problem and the processing information are sent to the artificial customer service for handling the after-sales problem.

[0133] In this step, if the problem classification confidence corresponding to the after-sales problem exceeds the preset confidence threshold, the system will directly handle the after-sales problem through the automatic handling module according to the problem classification result and the processing information. This way can quickly generate detailed handling schemes in the case of high confidence and send them to the customer in the form of formal notification, reducing waiting time and improving service efficiency and customer satisfaction. Automatic handling not only improves the response speed, but also reduces the need for human intervention and reduces operating costs.

[0134] Otherwise, when the problem classification confidence is lower than the preset threshold, the system will send the problem classification result and the preliminary generated processing information to the artificial customer service team for further review and handling. Based on these information, the artificial customer service can ensure that complex or uncertain problems are paid attention to and properly solved by professional personnel, avoid misjudgment or improper handling due to system limitations, provide more personalized and accurate services, and thus comprehensively improve the quality of after-sales service and customer experience.

[0135] As can be seen from the above, the intelligent e-commerce customer service after-sales processing method proposed in the embodiments of the present application can provide the following technical effects.

[0136] The embodiments of the present application effectively improve the accuracy and confidence of problem classification by comprehensively analyzing text, image and dialogue history data and combining importance ratio for interactive feature enhancement and fusion processing, and reduce misjudgment. The automatic handling module can quickly generate detailed handling schemes and directly send them to the customer in the case of high confidence, reducing the need for human intervention, speeding up the response, improving the service efficiency and customer satisfaction. For complex or low confidence problems, the system sends preliminary handling suggestions to the artificial customer service for further review, ensuring that the problem is paid attention to and properly solved by professional personnel, avoiding customer dissatisfaction due to system limitations. This method not only optimizes user experience and reduces operating costs, but also improves resource utilization efficiency, effectively improves the quality and efficiency of after-sales service, and brings effective technical effects and economic benefits to enterprises and consumers.

[0137] Secondly, referring to Figure 3The embodiment of the application provides a kind of intelligent e-commerce customer service after-sales processing system, including data acquisition module, feature processing module, weight adjustment module, feature interaction module and after-sales processing module.

[0138] Data acquisition module is used to obtain text data, image data, conversation history data and compensation mechanism associated with after-sales problems.

[0139] Feature processing module is used to feature process text data and image data, to obtain text features and image features.

[0140] Weight adjustment module is used to obtain the importance ratio between text features and image features according to conversation history data.

[0141] Feature interaction module is used to combine the importance ratio, to interactively feature enhance and fuse text features and image features, to obtain problem classification results and problem classification confidence.

[0142] After-sales processing module is used to generate processing information corresponding to after-sales problems according to problem classification results, problem classification confidence and compensation mechanism;According to the problem classification result, problem classification confidence and processing information corresponding to the after-sales problem, the after-sales problem is handled.

[0143] In some embodiments of the application, the after-sales processing module includes an automated processing unit and a manual processing unit.

[0144] Automated processing unit is used to automatically handle after-sales problems according to problem classification results and processing information corresponding to after-sales problems when problem classification confidence corresponding to after-sales problems exceeds a preset confidence threshold.

[0145] Manual processing unit is used to send problem classification results and processing information corresponding to after-sales problems to manual customer service when problem classification confidence corresponding to after-sales problems does not exceed a preset confidence threshold, to facilitate manual customer service to handle after-sales problems.

[0146] Further, with reference to Figure 4 The embodiment of the application provides an intelligent e-commerce customer service after-sales processing device, including processor and memory, memory is used to store program.When program is executed by processor, make processor realize the foregoing intelligent e-commerce customer service after-sales processing method.

[0147] In addition, the embodiment of the application provides a computer readable storage medium, wherein the processor executable program is stored, and the processor executable program is used to realize the foregoing intelligent e-commerce customer service after-sales processing method when executed by processor.

[0148] Similarly, the contents of the above method embodiments are applicable to the system embodiment, the device embodiment and the storage medium embodiment, the system embodiment, the device embodiment and the storage medium embodiment specifically realize the functions same as the above method embodiments, and achieve the beneficial effects same as the above method embodiments.

[0149] In some alternative embodiments, the functions / operations mentioned in the block diagrams can not occur in the order mentioned in the operation diagrams. For example, depending on the functions / operations involved, two blocks shown in succession can actually be executed substantially concurrently or the blocks can sometimes be executed in reverse order. Additionally, the embodiments presented and described in the flow diagrams are only examples. The steps presented and / or described in the flow diagrams are not exclusive and other steps can be included or excluded without departing from the scope of the disclosure. The steps presented and / or described in the flow diagrams can be modified in various ways without departing from the scope of the disclosure. Furthermore, embodiments from two or more of the methods can be combined to form additional methods without departing from the scope of the disclosure. Alternative embodiments are within the scope of the disclosure. For instance, one or more of the steps can be performed in differing order or concurrently. As another example, one or more of the described functions / operations can be performed in parallel. Also, the disclosed methods are not limited to the order of the operations presented and / or described in the flow diagrams. Alternative embodiments are within the scope of the disclosure.

[0150] Further, while the present application has been described in the context of functional modules, it is to be understood that one or more of the functions and / or features can be integrated in a single physical device and / or software module, or one or more functions and / or features can be implemented in separate physical devices or software modules. It is also to be understood that detailed discussion of the actual implementation of each module is unnecessary to an understanding of the present application. Rather, the actual implementation is within the routine of an engineer's knowledge given the property, function and internal relationships of the various functional modules disclosed herein. Accordingly, the present application is not limited to purely hardware or software implementations, but rather encompasses hybrid implementations wherein all or some of the functions are implemented in hardware and / or software. It is also to be understood that the disclosed specific concepts are merely illustrative and are not intended to limit the scope of the present application, which is defined by the appended claims and their equivalents.

[0151] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the present application that essentially contribute to the prior art or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of programs for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0152] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a list of ordered steps for implementing logical functions, and can be embodied in any computer readable medium for use by a program execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can take programs from a program execution system, device or apparatus and execute them) or in conjunction with these program execution systems, devices or apparatus. For the purpose of this specification, "computer readable medium" can be any device that can contain, store, communicate, propagate or transport programs for use by program execution systems, devices or apparatus or in conjunction with these program execution systems, devices or apparatus.

[0153] More specific examples (non-exhaustive list) of the computer readable medium include the following: an electrical connection having one or more wires (electrical devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CD ROM). In addition, the computer readable medium can even be paper or other suitable medium on which the program can be printed, because the program can be obtained electronically, for example, by optical scanning of the paper or other medium, followed by editing, interpreting or otherwise processing, if necessary, in other suitable ways, to be stored in a computer memory.

[0154] It should be understood that various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable

[0155] In the above description of the present application, the description referring to the terms "one embodiment", "another embodiment" or "certain embodiments" or the like means that a particular feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative appearances of the above-described terms in various places in the specification are not necessarily referred to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0156] Although the embodiments of the present application have been shown and described, it would be appreciated by those skilled in the art that changes, modifications, alternatives and variations to these embodiments could be made without departing from the principles and spirit of the application, the scope of which is defined in the appended claims and their equivalents.

[0157] The above is a specific description of the preferred embodiments of the present application, but the present application is not limited to the described embodiments, and those skilled in the art can make various equivalent modifications or replacements without departing from the spirit of the present application, and these equivalent modifications or replacements are all included in the scope defined by the claims of the present application.

Claims

1. An intelligent e-commerce customer service after-sales processing method, characterized in that, The method comprises the following steps: obtaining text data, image data, dialogue history data and compensation mechanism associated with an after-sales problem; performing feature processing on the text data and the image data to obtain text features and image features; performing feature processing on the dialogue history data by using a preset text processing model to obtain a dialogue history vector; calculating an importance parameter corresponding to the text features and an importance parameter corresponding to the image features according to the dialogue history vector, and obtaining an importance ratio between the text features and the image features; wherein the importance parameter corresponding to the text features satisfies the following formula: ； wherein, denotes an importance parameter corresponding to the text feature, denotes a dialogue history vector, denotes a random weight matrix corresponding to the text feature, denotes a random weight matrix corresponding to the image feature; an importance parameter corresponding to the image feature is denoted as: ; performing linear transformation on the text features and the image features according to the importance ratio to obtain weighted text features and weighted image features of the same dimension; obtaining self-enhanced text features by using a self-attention mechanism of a deep learning model according to the weighted text features; obtaining self-enhanced image features by using a self-attention mechanism of a deep learning model according to the weighted image features; obtaining cross-modal enhanced text features and cross-modal enhanced image features by using a cross-modal attention mechanism of a deep learning model according to the weighted text features and the weighted image features; obtaining a problem classification result and a problem classification confidence by using a multi-layer perception model according to the self-enhanced text features, the self-enhanced image features, the cross-modal enhanced text features and the cross-modal enhanced image features; generating processing information corresponding to the after-sales problem according to the problem classification result, the problem classification confidence and the compensation mechanism; if the problem classification confidence corresponding to the after-sales problem exceeds a preset confidence threshold, automatically processing the after-sales problem according to the problem classification result and the processing information corresponding to the after-sales problem; otherwise, sending the problem classification result and the processing information corresponding to the after-sales problem to an artificial customer service for processing the after-sales problem by the artificial customer service. 2.The intelligent e-commerce customer service after-sale processing method according to claim 1, characterized in that, The feature processing on the text data and the image data to obtain text features and image features comprises: performing feature processing on the text data by using a preset text processing model to obtain the text features; performing feature processing on the image data by using a preset image processing model to obtain the image features.

3. An intelligent e-commerce customer service after-sales processing system, characterized in that, The method comprises a data acquisition module, a feature processing module, a weight adjustment module, a feature interaction module and an after-sales processing module; The data acquisition module is configured to obtain text data, image data, dialogue history data and compensation mechanism associated with an after-sales problem; The feature processing module is configured to perform feature processing on the text data and the image data to obtain text features and image features; The weight adjustment module is configured to obtain an importance ratio between the text features and the image features according to the dialogue history data, comprising: performing feature processing on the dialogue history data by using a preset text processing model to obtain a dialogue history vector; calculating an importance parameter corresponding to the text features and an importance parameter corresponding to the image features according to the dialogue history vector, and obtaining an importance ratio between the text features and the image features; wherein the importance parameter corresponding to the text features satisfies the following formula: ； wherein, denotes an importance parameter corresponding to the text feature, denotes a dialogue history vector, denotes a random weight matrix corresponding to the text feature, denotes a random weight matrix corresponding to the image feature; an importance parameter corresponding to the image feature is denoted as: ; The feature interaction module is configured to combine the importance ratio, perform interactive feature enhancement processing and fusion processing on the text features and the image features, and obtain a problem classification result and a problem classification confidence, including: The text features and the image features are linearly transformed in combination with the importance ratio to obtain weighted text features and weighted image features in the same dimension; According to the weighted text features, a self-enhanced text feature is obtained by using a self-attention mechanism of a deep learning model; according to the weighted image features, a self-enhanced image feature is obtained by using the self-attention mechanism of the deep learning model; according to the weighted text features and the weighted image features, a cross-modal enhanced text feature and a cross-modal enhanced image feature are obtained by using a cross-modal attention mechanism of the deep learning model; and according to the self-enhanced text features, the self-enhanced image features, the cross-modal enhanced text feature and the cross-modal enhanced image feature, the problem classification result and the problem classification confidence are obtained by using a multi-layer perception model. The after-sales processing module is configured to generate processing information corresponding to the after-sales problem according to the problem classification result, the problem classification confidence and the compensation mechanism; if the problem classification confidence corresponding to the after-sales problem exceeds a preset confidence threshold, the after-sales problem is automatically processed according to the problem classification result and the processing information corresponding to the after-sales problem; otherwise, the problem classification result and the processing information corresponding to the after-sales problem are sent to an artificial customer service for processing the after-sales problem.

4. An intelligent e-commerce customer service after-sales processing device, characterized in that, The processor and the memory are included, and the memory is configured to store a program; when the program is executed by the processor, the processor implements the intelligent e-commerce customer service after-sales processing method in any one of claims 1 to 2.

5. A computer readable storage medium having stored therein a program which is executable by a processor, characterized in that, The program executable by the processor, when executed by the processor, is configured to implement the intelligent e-commerce customer service after-sales processing method in any one of claims 1 to 2.

Images