A cross-project experience transfer method for intelligent customer service combined with transfer learning

Abstract

The present invention discloses a cross-project experience transfer method for intelligent customer service combined with transfer learning, S1, generating a source project feature set; S2, generating a source project optimization model including a shared feature layer; S3, globally optimizing the shared feature layer parameters in the source project optimization model through Bayesian optimization; S4, generating an optimized target project feature set; S5, building a target project model; S6, fine-tuning the target project model; S7, performing semantic consistency evaluation, response accuracy evaluation and business logic matching evaluation on the fine-tuned target project model on the target project data set; S8, deploying the target project model that has passed performance verification to the target project intelligent customer service system. The present invention realizes efficient deployment of a cross-project intelligent customer service system while significantly improving the migration efficiency and service quality of the model, and can be widely used in intelligent customer service scenarios in multiple industries.

Description

Technical Field

[0001] The present invention relates to the field of intelligent customer service technology, and in particular to an intelligent customer service cross-project experience transfer method combined with transfer learning. Background Art

[0002] With the rapid development of artificial intelligence and natural language processing technologies, intelligent customer service systems have been widely used in various industries. Intelligent customer service systems improve the service efficiency of enterprises through semantic analysis and automated responses. In actual applications, there are significant differences in business needs, user habits, and language expressions between different projects, which poses new challenges to the deployment and expansion of intelligent customer service systems.

[0003] At present, most intelligent customer service systems often need to independently build and train models in different projects. Traditional methods usually train models based on data from a single project, and use deep learning algorithms to recognize user intent and build response logic. Although this method can provide more accurate services for specific projects, it also has obvious technical defects: First, independent training models for a single project requires a lot of data labeling work and computing resources, and the training cycle is long and the cost is high; second, for projects with similar business logic, traditional methods cannot effectively utilize the built models and knowledge, and need to train from scratch, resulting in duplication of work; in addition, there is heterogeneity in the distribution of knowledge between different projects, and it is difficult for traditional models to maintain the stability of semantic understanding capabilities during the migration process, which is prone to cause "negative transfer" problems, that is, the performance of the migrated model in the target project decreases.

[0004] In recent years, transfer learning technology has gradually been introduced into the field of intelligent customer service systems to alleviate the problem of repeated training in multi-project deployment. By extracting shared features from the source project and migrating them to the target project, transfer learning can significantly reduce the training data requirements of the target project. However, the existing transfer learning methods still have the following problems in application: On the one hand, the existing methods lack dynamic adaptation capabilities when dealing with the semantic and business differences between the source project and the target project, and cannot adjust the model parameters according to the specific needs of the target project; on the other hand, the migration process usually fails to comprehensively consider multi-dimensional optimization goals, semantic consistency, response accuracy, and business logic matching, resulting in limited migration effects.

[0005] In summary, the existing technologies have significant deficiencies in adaptability, efficiency and performance optimization in multi-project transfer learning, and it is difficult to meet the needs of rapid deployment and precise service of intelligent customer service systems. Therefore, there is an urgent need for a method that can dynamically adapt to heterogeneous knowledge, comprehensively optimize multi-dimensional performance and improve migration efficiency to overcome the above technical defects. Summary of the invention

[0006] One purpose of the present invention is to propose a method for cross-project experience transfer of intelligent customer service combined with transfer learning. The present invention realizes efficient deployment of cross-project intelligent customer service systems while significantly improving the migration efficiency and service quality of the model, and can be widely used in intelligent customer service scenarios in multiple industries.

[0007] According to an embodiment of the present invention, a method for intelligent customer service cross-project experience transfer combined with transfer learning includes the following steps:

[0008] S1. Obtain a source project dataset and a target project dataset, preprocess the source project dataset, and generate a source project feature set;

[0009] S2. Based on the source project feature set, a source project basic model is constructed using a deep learning algorithm, and trained on the source project dataset to generate a source project optimization model containing a shared feature layer;

[0010] S3. Globally optimize the shared feature layer parameters in the source project optimization model through Bayesian optimization;

[0011] S4, semantic expansion, synonym replacement and noise addition are performed on the target project data set to generate an optimized target project feature set;

[0012] S5. Building a target project model based on the optimized target project feature set and the optimized source project shared feature layer parameters;

[0013] S6. Fine-tune the target project model and use Bayesian optimization to dynamically adjust the weight distribution ratio between the shared feature layer and the target specific layer;

[0014] S7. Perform semantic consistency evaluation, response accuracy evaluation, and business logic matching evaluation on the fine-tuned target project model on the target project dataset, and further optimize the target project model parameters based on the verification results;

[0015] S8. Deploy the target project model that has passed performance verification to the target project intelligent customer service system, and combine the target project's knowledge base and business logic to implement the launch and operation of the intelligent customer service system.

[0016] Optionally, the S1 specifically includes:

[0017] S11. Collect source project data sets , and the source project dataset Perform data cleaning to remove missing values, incomplete data, and invalid samples to obtain the cleaned source project data set;

[0018] S12. Extract features from the cleaned source project data set and use natural language processing technology to convert the input data Convert to feature vector , get the source project data set after feature extraction;

[0019] S13: Label the source project data set after feature extraction, and label each feature vector according to the business needs of the source project. Corresponding tags Check and update to obtain the annotated source project dataset and generate the source project feature set for transfer learning ;

[0020] S14. Target project data set according to the data structure of the target project Perform format alignment processing to make the target project dataset consistent with the source project feature set compatible.

[0021] Optionally, the S2 specifically includes:

[0022] S21. Based on source project feature set All feature vectors Build the input matrix in index order :

[0023] ;

[0024] in, is the number of samples in the source project feature set, d is the feature dimension, is a set of real numbers, T is a transposed matrix;

[0025] S22, build a semantic understanding module, input matrix Each row of the feature vector Through embedding transformation and variant bidirectional long short-term memory network processing, semantic feature representation is formed ;

[0026] S23, build business logic modeling module, semantic features Mapping to business logic representation :

[0027] ;

[0028] in, Mapping parameter matrix for business logic, is the corresponding bias term, is a unidirectional semantic feature dimension, Represents dimensions for business logic, is a nonlinear activation function;

[0029] S24. Build a context modeling module to represent business logic through a temporal convolutional network Perform context feature modeling and generate context features :

[0030] ;

[0031] in, is a business logic sequence fragment of length T, and are the convolution parameters and bias terms, is the context feature dimension;

[0032] S25. Using contextual features Generate shared feature layer output :

[0033] ;

[0034] in, is the shared feature layer mapping parameter, is the corresponding bias term, is the dimension of the shared feature layer;

[0035] S26. Train the source project optimization model based on the source project feature set and define the source project loss function ;

[0036] S27. After completing the source project optimization model training, save the shared feature layer parameter set :

[0037] ;

[0038] in, Optimize the set of model parameters for the source project, is the set of shared feature layer parameters obtained through training.

[0039] Optionally, the variant bidirectional long short-term memory network dynamically adapts the source item knowledge by introducing a domain selection gating unit, and adds a domain selection gating factor in the forward LSTM unit:

[0040] ;

[0041] in, is the activation function, Select gating factors for the forward domain to dynamically adjust input features and the hidden state at the previous time step The impact on the current memory unit, Select the gated unit for transforming the hidden state in the forward domain The weight matrix of Select the gating unit for the forward domain to transform the current input features The weight matrix of The bias vector of the forward domain selection gating unit is used to balance the activation output of the domain selection gating unit;

[0042] Add domain selection gating factor in the backward LSTM unit:

[0043] ;

[0044] in, Select gating factors for the backward domain to dynamically adjust input features and the hidden state at the next time step The impact on the current memory unit, Select the gated unit for transforming the hidden state in the backward domain The weight matrix of Select the gated unit for transforming the current input features in the backward domain The weight matrix of A bias vector for the backward domain selection gating unit, used to balance the activation output of the domain selection gating unit;

[0045] Combined with the forward and backward domain selection gating operation, the generated forward semantic features and backward semantic features Merge into bidirectional semantic features:

[0046] ;

[0047] in, is the semantic feature representation after forward processing by the domain selection gating factor. It is the semantic feature representation after backward processing after the domain selection gating factor.

[0048] Optionally, the S3 specifically includes:

[0049] S31. Define the shared feature layer parameter set of the source project optimization model , initialize the Bayesian optimization search space , the Bayesian optimization search space contains the value ranges of all shared feature parameters;

[0050] S32, comprehensive semantic consistency index , Response accuracy index Business logic matching index Constructing the optimization objective function :

[0051] ;

[0052] in, is the current shared feature layer parameter value, is the weight coefficient of the indicator;

[0053] S33, optimize the objective function through Gaussian process Modeling and constructing predictive distributions , where D is the current observation data set, including several parameter values and its corresponding target value ;

[0054] S34. Define acquisition function To balance exploration and utilization, select the next parameter value and select the optimal parameter value based on the acquisition function:

[0055] ;

[0056] S35, Add the observation data set D, repeat the S31-S34 process until convergence, and finally obtain the global optimal source project shared feature layer parameters:

[0057] ;

[0058] The optimized shared feature layer parameters Stores the final shared feature parameters for the source project's optimized model.

[0059] Optionally, the S5 specifically includes:

[0060] S51. Define the target project feature set as , the target project feature set Build the input matrix in index order Serves as the input basis for the target project model;

[0061] S52: Initialize the shared feature layer parameter set of the target project model, and the optimized shared feature layer parameter set of the source project As a set of initialization parameters for the target project's shared feature layer , so that the shared feature layer knowledge of the source project can be transferred to the target project;

[0062] The input layer receives the target item input matrix ;

[0063] The shared feature layer is based on the initialization parameters Process input data to generate shared features ;

[0064] Target specific layers utilize target specific layer parameters Shared Features Mapping to target project task-specific feature representation ;

[0065] The output layer uses the output layer parameters Generate final predictions ;

[0066] S54, define the overall optimization parameter set of the target project model as , jointly optimize the parameter set of the target project model, through the loss function of the target task The minimization process updates the parameters;

[0067] S55. After the optimization is completed, the final optimization parameter set of the target project model is generated:

[0068] .

[0069] Optionally, the S6 specifically includes:

[0070] S61. Define the shared feature layer weight parameters of the target project model as and the target-specific layer weight parameters are , the weight distribution parameters satisfy the following constraints:

[0071] ;

[0072] in, Used to control the output ratio of the shared feature layer. It is used to control the output ratio of the target specific layer and dynamically adjust the ratio of the two through Bayesian optimization;

[0073] S62, based on the target project validation set Construct optimization indicators, where For the validation set input, To correspond to the true label, Index the validation set, and optimize the following indicators:

[0074] Semantic matching index , used to measure the semantic understanding ability of the target project model on the validation set;

[0075] Response accuracy index , used to measure the task completion ability of the target project model on the validation set;

[0076] S63. Define an optimization objective function, which performs weighted integration of semantic matching degree and response accuracy:

[0077] ;

[0078] in, , is the weight coefficient;

[0079] S64. Using Bayesian optimization to optimize weight parameters Dynamically adjust the search space for:

[0080] ;

[0081] By constructing the prediction distribution of the optimized objective function and selecting the acquisition function to select the optimal weight parameters;

[0082] S65. Update observation data set , repeat the optimization process until convergence, and obtain the global optimal weight parameter:

[0083] ;

[0084] S66. Using the optimized weight parameters The shared feature layer and target-specific layer outputs of the target project model are weighted together to generate the final prediction value ;

[0085] S67. After completing the fine-tuning training, a parameter set of the fine-tuned target project model is generated:

[0086] ;

[0087] in, , , are the optimization parameters for the shared feature layer, target-specific layer, and output layer, respectively.

[0088] The beneficial effects of the present invention are:

[0089] (1) The present invention introduces Bayesian optimization technology to dynamically adjust the weight allocation ratio of the shared feature layer and the target specific layer during the fine-tuning stage of the target project model, and adaptively optimizes the model parameters according to the semantic matching and response accuracy indicators of the target project verification set. Traditional methods usually use a fixed ratio to allocate weights, which makes it difficult to balance the adaptability of shared knowledge and target specific tasks. The dynamic weight adjustment mechanism of the present invention significantly improves the semantic understanding ability and response accuracy of the target project model for new tasks.

[0090] (2) The present invention combines semantic consistency, response accuracy, and business logic matching multi-dimensional optimization goals in the training and migration process of the source project optimization model, and globally adjusts the shared feature layer parameters through Bayesian optimization to ensure the knowledge generalization ability and task adaptability in the migration learning process. It can achieve dynamic trade-offs between multi-dimensional goals and reduce the performance degradation of the model caused by negative transfer during the migration process.

[0091] (3) The present invention constructs a target project model that is adapted to a heterogeneous data environment by using semantic expansion, synonym replacement, and noise addition preprocessing techniques on the target project data set, combined with optimized source project shared feature layer parameters. In the process of target project model construction and optimization, effective alignment of data distribution differences is achieved, which significantly improves the adaptability of the target project model in a heterogeneous data environment. The method of the present invention improves performance stability by 15% in cross-project scenarios with large data distribution differences, significantly shortens the training time of the target project model, and has been successfully applied to multiple actual scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0093] Figure 1 This is a flow chart of a method for intelligent customer service cross-project experience transfer combined with transfer learning proposed by the present invention. DETAILED DESCRIPTION

[0094] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0095] refer to Figure 1 , a method for intelligent customer service cross-project experience transfer combined with transfer learning, comprising the following steps:

[0096] S1. Obtain a source project dataset and a target project dataset, preprocess the source project dataset, and generate a source project feature set;

[0097] S2. Based on the source project feature set, a source project basic model is constructed using a deep learning algorithm, and trained on the source project dataset to generate a source project optimization model containing a shared feature layer;

[0098] S3. Globally optimize the shared feature layer parameters in the source project optimization model through Bayesian optimization;

[0099] S4, semantic expansion, synonym replacement and noise addition are performed on the target project data set to generate an optimized target project feature set;

[0100] S5. Building a target project model based on the optimized target project feature set and the optimized source project shared feature layer parameters;

[0101] S6. Fine-tune the target project model and use Bayesian optimization to dynamically adjust the weight distribution ratio between the shared feature layer and the target specific layer;

[0102] S7. Perform semantic consistency evaluation, response accuracy evaluation, and business logic matching evaluation on the fine-tuned target project model on the target project dataset, and further optimize the target project model parameters based on the verification results;

[0103] S8. Deploy the target project model that has passed performance verification to the target project intelligent customer service system, and combine the target project's knowledge base and business logic to implement the launch and operation of the intelligent customer service system.

[0104] In this implementation, S1 specifically includes:

[0105] S11. Collect source project data sets , and the source project dataset Perform data cleaning to remove missing values, incomplete data, and invalid samples to obtain the cleaned source project data set;

[0106] S12. Extract features from the cleaned source project data set and use natural language processing technology to convert the input data Convert to feature vector , get the source project data set after feature extraction;

[0107] S13: Label the source project data set after feature extraction, and label each feature vector according to the business needs of the source project. Corresponding tags Check and update to obtain the annotated source project dataset and generate the source project feature set for transfer learning ;

[0108] S14. Target project data set according to the data structure of the target project Perform format alignment processing to make the target project dataset consistent with the source project feature set compatible.

[0109] In this implementation, S2 specifically includes:

[0110] S21. Based on source project feature set All feature vectors Build the input matrix in index order :

[0111] ;

[0112] in, is the number of samples in the source project feature set, d is the feature dimension, is a set of real numbers, T is a transposed matrix;

[0113] S22, build a semantic understanding module, input matrix Each row of the feature vector Through embedding transformation and variant bidirectional long short-term memory network processing, semantic feature representation is formed ;

[0114] S23, build business logic modeling module, semantic features Mapping to business logic representation :

[0115] ;

[0116] in, Mapping parameter matrix for business logic, is the corresponding bias term, is a unidirectional semantic feature dimension, Represents dimensions for business logic, is a nonlinear activation function;

[0117] S24. Build a context modeling module to represent business logic through a temporal convolutional network Perform context feature modeling and generate context features :

[0118] ;

[0119] in, is a business logic sequence fragment of length T, and are the convolution parameters and bias terms, is the context feature dimension;

[0120] S25. Using contextual features Generate shared feature layer output :

[0121] ;

[0122] in, is the shared feature layer mapping parameter, is the corresponding bias term, is the dimension of the shared feature layer;

[0123] S26. Train the source project optimization model based on the source project feature set and define the source project loss function ;

[0124] S27. After completing the source project optimization model training, save the shared feature layer parameter set :

[0125] ;

[0126] in, Optimize the set of model parameters for the source project, is the set of shared feature layer parameters obtained through training.

[0127] In this implementation, the variant bidirectional long short-term memory network dynamically adapts the source project knowledge by introducing a domain selection gating unit, and adds a domain selection gating factor in the forward LSTM unit:

[0128] ;

[0129] in, is the activation function, Select gating factors for the forward domain to dynamically adjust input features and the hidden state at the previous time step The impact on the current memory unit, Select the gated unit for transforming the hidden state in the forward domain The weight matrix of Select the gating unit for the forward domain to transform the current input features The weight matrix of The bias vector of the forward domain selection gating unit is used to balance the activation output of the domain selection gating unit;

[0130] Add domain selection gating factor in the backward LSTM unit:

[0131] ;

[0132] in, Select gating factors for the backward domain to dynamically adjust input features and the hidden state at the next time step The impact on the current memory unit, Select the gated unit for transforming the hidden state in the backward domain The weight matrix of Select the gated unit for transforming the current input features in the backward domain The weight matrix of A bias vector for the backward domain selection gating unit, used to balance the activation output of the domain selection gating unit;

[0133] Combined with the forward and backward domain selection gating operation, the generated forward semantic features and backward semantic features Merge into bidirectional semantic features:

[0134] ;

[0135] in, is the semantic feature representation after forward processing by the domain selection gating factor. It is the semantic feature representation after backward processing after the domain selection gating factor.

[0136] In this implementation, S3 specifically includes:

[0137] S31. Define the shared feature layer parameter set of the source project optimization model , initialize the Bayesian optimization search space , the Bayesian optimization search space contains the value ranges of all shared feature parameters;

[0138] S32, comprehensive semantic consistency index , Response accuracy index Business logic matching index Constructing the optimization objective function :

[0139] ;

[0140] in, is the current shared feature layer parameter value, is the weight coefficient of the indicator;

[0141] S33, optimize the objective function through Gaussian process Modeling and constructing predictive distributions , where D is the current observation data set, including several parameter values and its corresponding target value ;

[0142] S34. Define acquisition function To balance exploration and utilization, select the next parameter value and select the optimal parameter value based on the acquisition function:

[0143] ;

[0144] S35, Add the observation data set D, repeat the S31-S34 process until convergence, and finally obtain the global optimal source project shared feature layer parameters:

[0145] ;

[0146] The optimized shared feature layer parameters Stores the final shared feature parameters for the source project's optimized model.

[0147] In this implementation, S5 specifically includes:

[0148] S51. Define the target project feature set as , the target project feature set Build the input matrix in index order Serves as the input basis for the target project model;

[0149] S52: Initialize the shared feature layer parameter set of the target project model, and the optimized shared feature layer parameter set of the source project As a set of initialization parameters for the target project's shared feature layer , so that the shared feature layer knowledge of the source project can be transferred to the target project;

[0150] S53, construct a target project model, the target project model includes an input layer, a shared feature layer, a target specific layer and an output layer:

[0151] The input layer receives the target item input matrix ;

[0152] The shared feature layer is based on the initialization parameters Process input data to generate shared features ;

[0153] Target specific layers utilize target specific layer parameters Shared Features Mapping to target project task-specific feature representation ;

[0154] The output layer uses the output layer parameters Generate final predictions ;

[0155] S54, define the overall optimization parameter set of the target project model as , jointly optimize the parameter set of the target project model, through the loss function of the target task The minimization process updates the parameters;

[0156] S55. After the optimization is completed, the final optimization parameter set of the target project model is generated:

[0157] .

[0158] In this implementation, S6 specifically includes:

[0159] S61. Define the shared feature layer weight parameters of the target project model as and the target-specific layer weight parameters are , the weight distribution parameters satisfy the following constraints:

[0160] ;

[0161] in, Used to control the output ratio of the shared feature layer. It is used to control the output ratio of the target specific layer and dynamically adjust the ratio of the two through Bayesian optimization;

[0162] S62, based on the target project validation set Construct optimization indicators, where For the validation set input, To correspond to the true label, Index the validation set, and optimize the following indicators:

[0163] Semantic matching index , used to measure the semantic understanding ability of the target project model on the validation set;

[0164] Response accuracy index , used to measure the task completion ability of the target project model on the validation set;

[0165] S63. Define an optimization objective function, which performs weighted integration of semantic matching degree and response accuracy:

[0166] ;

[0167] in, , is the weight coefficient;

[0168] S64. Using Bayesian optimization to optimize weight parameters Dynamically adjust the search space for:

[0169] ;

[0170] By constructing the prediction distribution of the optimized objective function and selecting the acquisition function to select the optimal weight parameters;

[0171] S65. Update observation data set , repeat the optimization process until convergence, and obtain the global optimal weight parameter:

[0172] ;

[0173] S66. Using the optimized weight parameters The shared feature layer and target-specific layer outputs of the target project model are weighted together to generate the final prediction value ;

[0174] S67. After completing the fine-tuning training, a parameter set of the fine-tuned target project model is generated:

[0175] ;

[0176] in, , , are the optimization parameters for the shared feature layer, target-specific layer, and output layer, respectively.

[0177] Embodiment 1:

[0178] In order to verify the feasibility and superiority of the present invention in implementation, the embodiment selected two natural language processing dialogue datasets as experimental objects, namely MultiWOZ 2.1 (multi-domain dialogue dataset) and DSTC8 (dialogue state tracking challenge dataset). MultiWOZ 2.1 contains multi-domain dialogue information, including hotel reservations, restaurant inquiries and taxi dispatch, with a total of 10,438 dialogue records, annotated with user intentions, slots and value information. The DSTC8 dataset is mainly used to test the multi-domain state tracking capability of the dialogue system, containing 14,000 multi-round dialogue records, covering multiple complex business scenarios.

[0179] In the embodiment, the MultiWOZ 2.1 dataset is used as the source project dataset, and the DSTC8 dataset is used as the target project dataset to verify the feasibility and effectiveness of the intelligent customer service cross-project experience transfer method combining transfer learning and Bayesian optimization proposed in the present invention. The dataset is divided into a training set and a test set in a ratio of 8:2, and 10% of the training set is used as a validation set for model evaluation.

[0180] On the MultiWOZ 2.1 dataset, the user conversation text is segmented, stop words are removed, and synonyms are replaced to generate the source project feature set. A total of 500,000 feature vectors with a feature dimension of 768 are extracted. A similar processing method is used to generate the target project feature set for the DSTC8 dataset. A total of 100,000 feature vectors are extracted. The sample is expanded to 250,000 by adding noise and semantic expansion to make up for the problem of insufficient target project data.

[0181] A deep learning model including semantic understanding module, business logic modeling module and context modeling module was built on the MultiWOZ 2.1 dataset. The BERT pre-trained model was used to extract text embedding with an input dimension of 768. Semantic features were extracted through a bidirectional LSTM network with a semantic feature dimension of 512. The model was trained using the Adam optimizer with a learning rate set to 0.001, a batch size of 64, and a total of 50 training rounds. After training, Bayesian optimization was used to globally adjust the parameters of the shared feature layer. During the optimization process, semantic consistency, response accuracy, and business logic matching were comprehensively considered to finally generate an optimized model for the source project.

[0182] The optimized shared feature layer parameters of the source project are migrated to the target project model. The weight distribution ratio of the shared feature layer and the target specific layer is dynamically adjusted through Bayesian optimization to ensure that the model is suitable for the target task. The target project model is trained on the DSTC8 dataset using the same optimizer and training parameters for fine-tuning. The number of training rounds is 30, and the parameter ratio of the shared and specific layers is dynamically optimized in combination with the target validation set.

[0183] In order to comprehensively evaluate the effectiveness of the method of the present invention, the following indicators are used:

[0184] Semantic consistency score: measures the semantic similarity between the predicted result and the true value, calculated using cosine similarity.

[0185] Response accuracy: measures whether the system correctly generates the response expected by the user.

[0186] Business logic matching: measures whether the model output conforms to the business logic rules.

[0187] The method of the present invention is comprehensively compared with the traditional single training method, and the experimental results are shown in Table 1:

[0188] Table 1 Comparison of experimental results between the method of the present invention and the traditional single training method

[0189] ;

[0190] As can be seen from Table 1, the source project model performs better than the traditional method in semantic consistency score and business logic matching, which are improved by 3.1% and 4.4% respectively after optimization. In the target project, the method of the present invention significantly improves the semantic consistency score (6.3%), response accuracy (7.2%) and business logic matching (7.7%) through shared feature migration and Bayesian optimization. In addition, the training time of the method of the present invention on the target project is shortened by more than 30%, and the performance on the verification set is more stable, which fully proves the superiority and practical application value of the method of the present invention.

[0191] This embodiment verifies the practical application capability and technical effect of the present invention through public data sets, fully solves the problems of long cross-project model training cycle, low migration efficiency, and poor adaptability of target tasks existing in traditional methods, and provides an efficient and reliable solution for the cross-scenario deployment of intelligent customer service systems.

[0192] The present invention introduces Bayesian optimization technology to dynamically adjust the weight allocation ratio of the shared feature layer and the target specific layer in the fine-tuning stage of the target project model, and adaptively optimizes the model parameters according to the semantic matching and response accuracy indicators of the target project verification set. Traditional methods usually use a fixed ratio to allocate weights, which makes it difficult to balance the adaptability of shared knowledge and target specific tasks. The dynamic weight adjustment mechanism of the present invention significantly improves the semantic understanding ability and response accuracy of the target project model for new tasks.

[0193] The present invention combines the multi-dimensional optimization goals of semantic consistency, response accuracy and business logic matching during the training and migration process of the source project optimization model, and globally adjusts the shared feature layer parameters through Bayesian optimization to ensure the knowledge generalization ability and task adaptability in the transfer learning process. It can achieve dynamic trade-offs between multi-dimensional goals and reduce the performance degradation of the model due to negative migration during the migration process.

[0194] The present invention constructs a target project model adapted to a heterogeneous data environment through semantic expansion, synonym replacement and noise addition preprocessing techniques for the target project data set, combined with optimized source project shared feature layer parameters, and achieves effective alignment of data distribution differences during the target project model construction and optimization process, thereby significantly improving the adaptability of the target project model in a heterogeneous data environment. The method of the present invention improves performance stability by 15% in cross-project scenarios with large data distribution differences, significantly shortens the training time of the target project model, and has been successfully applied to multiple actual scenarios.

[0195] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for intelligent customer service cross-project experience transfer combined with transfer learning, characterized in that: The steps include: S1. Obtain a source project dataset and a target project dataset, preprocess the source project dataset, and generate a source project feature set; S2. Based on the source project feature set, a source project basic model is constructed using a deep learning algorithm, and trained on the source project dataset to generate a source project optimization model containing a shared feature layer; S3. Globally optimize the shared feature layer parameters in the source project optimization model through Bayesian optimization; S4, semantic expansion, synonym replacement and noise addition are performed on the target project data set to generate an optimized target project feature set; S5. Building a target project model based on the optimized target project feature set and the optimized source project shared feature layer parameters; S6. Fine-tune the target project model and use Bayesian optimization to dynamically adjust the weight distribution ratio between the shared feature layer and the target specific layer; S7. Perform semantic consistency evaluation, response accuracy evaluation, and business logic matching evaluation on the fine-tuned target project model on the target project dataset, and further optimize the target project model parameters based on the verification results; S8. Deploy the target project model that has passed performance verification to the target project intelligent customer service system, and combine the target project's knowledge base and business logic to implement the launch and operation of the intelligent customer service system.

2. According to claim 1, a method for intelligent customer service cross-project experience transfer combined with transfer learning is characterized in that: The S1 specifically includes: S11. Collect source project data sets , and the source project dataset Perform data cleaning to remove missing values, incomplete data, and invalid samples to obtain the cleaned source project data set; S12. Extract features from the cleaned source project data set and use natural language processing technology to convert the input data Convert to feature vector , get the source project data set after feature extraction; S13: Label the source project data set after feature extraction, and label each feature vector according to the business needs of the source project. Corresponding tags Check and update to obtain the annotated source project dataset and generate the source project feature set for transfer learning ; S14. Target project data set according to the data structure of the target project Perform format alignment processing to make the target project dataset consistent with the source project feature set compatible.

3. According to claim 1, a method for intelligent customer service cross-project experience transfer combined with transfer learning is characterized in that: The S2 specifically includes: S21. Based on source project feature set All feature vectors Build the input matrix in index order : ; in, is the number of samples in the source project feature set, d is the feature dimension, is a set of real numbers, T is a transposed matrix; S22, build a semantic understanding module, input matrix Each row of the feature vector Through embedding transformation and variant bidirectional long short-term memory network processing, semantic feature representation is formed ; S23, build business logic modeling module, semantic features Mapping to business logic representation : ; in, Mapping parameter matrix for business logic, is the corresponding bias term, is a unidirectional semantic feature dimension, Represents dimensions for business logic, is a nonlinear activation function; S24. Build a context modeling module to represent business logic through a temporal convolutional network Perform context feature modeling and generate context features : ; in, is a business logic sequence fragment of length T, and are the convolution parameters and bias terms, is the context feature dimension; S25. Using contextual features Generate shared feature layer output : ; in, is the shared feature layer mapping parameter, is the corresponding bias term, is the dimension of the shared feature layer; S26. Train the source project optimization model based on the source project feature set and define the source project loss function ; S27. After completing the source project optimization model training, save the shared feature layer parameter set : ; in, Optimize the set of model parameters for the source project, is the set of shared feature layer parameters obtained through training.

4. According to claim 3, a method for intelligent customer service cross-project experience transfer combined with transfer learning is characterized in that: The variant bidirectional long short-term memory network dynamically adapts the source project knowledge by introducing a domain selection gating unit, and adds a domain selection gating factor in the forward LSTM unit: ; in, is the activation function, Select gating factors for the forward domain to dynamically adjust input features and the hidden state at the previous time step The impact on the current memory unit, Select the gated unit for transforming the hidden state in the forward domain The weight matrix of Select the gating unit for the forward domain to transform the current input features The weight matrix of The bias vector of the forward domain selection gating unit is used to balance the activation output of the domain selection gating unit; Add domain selection gating factor in the backward LSTM unit: ; in, Select gating factors for the backward domain to dynamically adjust input features and the hidden state at the next time step The impact on the current memory unit, Select the gated unit for transforming the hidden state in the backward domain The weight matrix of Select the gated unit for transforming the current input features in the backward domain The weight matrix of A bias vector for the backward domain selection gating unit, used to balance the activation output of the domain selection gating unit; Combined with the forward and backward domain selection gating operation, the generated forward semantic features and backward semantic features Merge into bidirectional semantic features: ; in, is the semantic feature representation after forward processing by the domain selection gating factor. It is the semantic feature representation after backward processing after the domain selection gating factor.

5. According to claim 1, a method for intelligent customer service cross-project experience transfer combined with transfer learning is characterized in that: The S3 specifically includes: S31. Define a set of shared feature layer parameters for the source project optimization model , initialize the Bayesian optimization search space , the Bayesian optimization search space contains the value ranges of all shared feature parameters; S32, comprehensive semantic consistency index , Response accuracy index Business logic matching index Constructing the optimization objective function : ; in, is the current shared feature layer parameter value, is the weight coefficient of the indicator; S33, optimize the objective function through Gaussian process Modeling and constructing predictive distributions , where D is the current observation data set, including several parameter values and its corresponding target value ; S34. Define acquisition function To balance exploration and utilization, select the next parameter value and select the optimal parameter value based on the acquisition function: ; S35, Add the observation data set D, repeat the S31-S34 process until convergence, and finally obtain the global optimal source project shared feature layer parameters: ; The optimized shared feature layer parameters Stores the final shared feature parameters for the source project's optimized model.

6. According to claim 1, a method for intelligent customer service cross-project experience transfer combined with transfer learning is characterized in that: The S5 specifically includes: S51. Define the target project feature set as , the target project feature set Build the input matrix in index order Serves as the input basis for the target project model; S52: Initialize the shared feature layer parameter set of the target project model, and the optimized shared feature layer parameter set of the source project As a set of initialization parameters for the target project's shared feature layer , so that the shared feature layer knowledge of the source project can be transferred to the target project; S53, construct a target project model, the target project model includes an input layer, a shared feature layer, a target specific layer and an output layer: The input layer receives the target item input matrix ; The shared feature layer is based on the initialization parameters Process input data to generate shared features ; Target specific layers utilize target specific layer parameters Shared Features Mapping to target project task-specific feature representation ; The output layer uses the output layer parameters Generate final predictions ; S54, define the overall optimization parameter set of the target project model as , jointly optimize the parameter set of the target project model, through the loss function of the target task The minimization process updates the parameters; S55. After the optimization is completed, the final optimization parameter set of the target project model is generated: 。 7. According to claim 1, a method for intelligent customer service cross-project experience transfer combined with transfer learning is characterized in that: The S6 specifically includes: S61. Define the shared feature layer weight parameters of the target project model as and the target-specific layer weight parameters are , the weight distribution parameters satisfy the following constraints: ； in, Used to control the output ratio of the shared feature layer. It is used to control the output ratio of the target specific layer and dynamically adjust the ratio of the two through Bayesian optimization; S62, based on the target project validation set Construct optimization indicators, where For the validation set input, To correspond to the true label, Index the validation set, and optimize the following indicators: Semantic matching index , used to measure the semantic understanding ability of the target project model on the validation set; Response accuracy index , used to measure the task completion ability of the target project model on the validation set; S63. Define an optimization objective function, which performs weighted integration of semantic matching degree and response accuracy: ; in, , is the weight coefficient; S64. Using Bayesian optimization to optimize weight parameters Dynamically adjust the search space for: ; By constructing the prediction distribution of the optimized objective function and selecting the acquisition function to select the optimal weight parameters; S65. Update observation data set , repeat the optimization process until convergence, and obtain the global optimal weight parameter: ; S66. Using the optimized weight parameters The shared feature layer and target-specific layer outputs of the target project model are weighted together to generate the final prediction value ; S67. After completing the fine-tuning training, a parameter set of the fine-tuned target project model is generated: ; in, , , are the optimization parameters for the shared feature layer, target-specific layer, and output layer, respectively.

Images