Customer personalized service system and method based on multi-modal data fusion

Abstract

The invention relates to the field of user portraits, in particular to a customer personalized service system and method based on multi-modal data fusion, and the system comprises a multi-source data module, a feature vector module, a node label module, a task aggregation module and a service prediction module, the feature vector module is used for extracting user feature vectors, the node label module is used for capturing homogeneous nodes, the task aggregation module is used for clustering personalized label vectors, and the service prediction module is used for outputting a probability value of executing a current task by a client. The timeliness and integrity of the data are ensured, the user portrait precision and the recognition accuracy of the user portrait model are improved, accurate allocation and personalized prediction of customer resources are realized, the accuracy of customer personalized label classification is improved, and the precision of a customer personalized recommendation algorithm is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of user profiling, in particular to a customer personalized service system and method based on multi-modal data fusion. BACKGROUND

[0002] CRM (Customer Relationship Management) system is a customer personalized management system relying on user profiling technology, which builds a user model based on real data, models the user's interest preferences, and can convert customer data into better predictions through technical means to recommend products or services that better meet customer needs. Common user profiling techniques determine user attribute labels by extracting dependency relationships within user operations, and then assign user types by labels; Label-based user profiling models are simple to build, but still have problems such as insufficient modeling accuracy, insufficient extraction of long-distance character interaction relationships, and overload of multi-source heterogeneous data. When facing non-labeled specific task decisions, the generalization ability is low, the anti-overfitting interference ability is poor, and it is difficult to make accurate decisions under high-dimensional data. Conventional numerical or text-based user profiling models also have problems such as insufficient use of multi-source data and poor data processing ability in an environment with a surge in business data.

[0003] In addition, conventional user profiling is based on the mining of massive data, and when there is insufficient directly obtained data, there is no other data source, which may result in missing model content, and user personalized data represented in character structure also has the problem of information loss in the processing process, so the data source and data processing process of the model need to be further improved. SUMMARY

[0004] The purpose of the present application is to provide a customer personalized service system and method based on multi-modal data fusion to solve the problems raised in the background art.

[0005] To solve the above technical problems, the present application provides the following technical solution: a customer personalized service system based on multi-modal data fusion, comprising: a multi-source data module, a feature vector module, a node label module, a task aggregation module and a service prediction module; The multi-source data module is used to set a prediction task target, publish user communication logs from each source to a pre-set Topic based on asynchronous publishing and subscribing from each business system, subscribe to the Topic to obtain data flow, and build a multi-source database after pre-cleaning, input the multi-source data into a Transformer network with a reconstructed attention layer, and extract structured information from each data record, including time, behavior and task action, and build structured triple data; The feature vector module is used to construct a dynamic directed acyclic graph (DAG) from all triples of data pointing to the same task target. The extracted task entities are used as nodes of the graph network, user attributes are defined as node attributes, and association relationships are used as directed edges connecting the nodes. The graph neural network is used for message passing to map the task entities and association relationships of the DAG to a low-dimensional vector space. The user features related to the task target are extracted through the MFEM model and transformed into vector representations to obtain the user feature vector. The node labeling module is used to fuse the elements in the user feature vector with the embedding vector of the corresponding task entity in the directed acyclic graph, and output a sparse vector. The sparse vector is projected onto the dense vector through a fully connected layer to obtain the latent vector. The latent vector is learned using the TGCL model, and the latent vector nodes are connected to the graph network nodes in chronological order to obtain the temporal graph. The GCN encoder learns the temporal graph representation, and the graph convolutional network is used to aggregate the adjacent information of each hop to capture homogeneous nodes. The task aggregation module is used to extract independent data related to the user and the current task based on homogeneous nodes. The firefly algorithm is used to determine the K value and cluster center in the K-means clustering process. The clustering result is output as personalized data. The personalized data of the user is converted into a low-dimensional representation through the label embedding layer. A three-layer learner structure is used to aggregate the low-dimensional data according to the user feature vector. The loss function is reconstructed to compensate for information loss and obtain personalized label vectors. The service prediction module is used to calculate the similarity of the task vector among all user personality tag vectors, take the preset number of vectors with the highest similarity as neighbor vectors, input the current user's personality tag vector into the convolutional neural network aggregator, aggregate the neighbor vectors in the task neighborhood according to the similarity order to obtain the current user's task vector, concatenate the personality tag vector and the task vector through a fully connected layer, perform an inner product operation on the concatenated vector, and output the probability value through an activation function to obtain the prediction result of the customer executing the current task.

[0006] Furthermore, the multi-source data module includes: an asynchronous acquisition unit and an information extraction unit; The asynchronous acquisition unit is used to obtain JSON format communication logs from the business system through a lightweight SDK and publish them to a preset topic in the message queue. The communication logs include: user ID, timestamp, task behavior, session ID and message content. The information extraction unit is used to associate multi-source log messages within a time window through a Transformer network to construct a structured triple containing task entities, association relationships, and user attributes.

[0007] Furthermore, the feature vector module includes: a graph neural network unit and a vector transformation unit; The graph neural network unit is used to connect user triples in chronological order to construct a directed acyclic graph, use a directed GNN or a temporal GNN for message passing, and update the graph structure representation by aggregating parent and child task information; The vector transformation unit is used to pool the task representation of all nodes, compress the graph size using the pooling layer, and extract task-related user features using the MFEM multi-view feature extraction and fusion model. The user features are input into the cross network and the output is a feature vector.

[0008] Furthermore, the node labeling module includes: a feature learning unit and a node aggregation unit; The feature learning unit is used to perform attention-weighted fusion of user feature vectors and embedding vectors of task entities in a directed acyclic graph to generate cross features; The node aggregation unit is used to map the feature vector after cross-feature fusion to a high-dimensional space using feature hashing, and project it onto a dense vector to obtain a latent vector.

[0009] Furthermore, the task aggregation module includes: a network aggregation unit, a learner unit, and a K-value clustering unit; The network aggregation unit is used to input the latent vector sequence into the TGCL time-series graph contrastive learning model, and output the time-series graph through contrastive learning and dynamic graph topology. The learner unit is used to set the learning time sequence graph of the GCN encoder, add similarity constraints between the node feature and the task vector when updating the node feature, and filter task-related data; The K-value clustering unit is used to employ the variable step size adaptive firefly algorithm, which uses the firefly's position code as the cluster center and the reciprocal of the intra-cluster distance as the brightness function to evaluate the clustering quality under different K values ​​and output the optimal K value and cluster center.

[0010] Furthermore, the service prediction module includes: a feature interaction unit, a concatenated inner product unit, and a behavior prediction unit; The feature interaction unit is used to aggregate the personalized tag vectors and calculate the cosine similarity between the current user's personalized tag vector and the personalized tag vectors of all users in the set task neighborhood. The cascaded inner product unit is used to select adjacent vectors according to similarity, arrange them into a matrix, input the matrix into a one-dimensional convolutional neural network for convolution and pooling, and output the task vector. The behavior prediction unit is used to concatenate the personality tag vector and the task vector. After nonlinear transformation and dimension alignment through a fully connected layer, it performs an element-wise inner product. The inner product result yields the interaction vector. The interaction vector is input into the Sigmoid activation function, which outputs the probability that the user will perform the set task.

[0011] A method for providing personalized customer services based on multimodal data fusion includes the following steps: Step S1. Obtain user communication logs from various sources based on publish-subscribe, clean them to obtain multi-source data, input the multi-source data into the Transformer network that reconstructs the attention layer to extract structured information, and construct structured triplet data; Step S2. Connect the triples pointing to the same task objective in chronological order to construct a directed acyclic graph. Map the task entities and relationships in the directed acyclic graph to a low-dimensional vector space. Extract user features related to the task objective through the MFEM model to obtain user feature vectors. Step S3. The user feature vector is fused with the embedding vector of the task entity to obtain a sparse vector. The sparse vector is projected onto the dense vector through a fully connected layer to obtain a latent vector. The TGCL model learns the latent vector. The TGCL model is a temporal graph contrastive learning model. The latent vector nodes are connected to the graph network nodes in chronological order to obtain a temporal graph. Step S4. Use a graph convolutional network to aggregate the adjacent information of each hop in the time series graph, capture homogeneous nodes, extract independent data related to the current task based on homogeneous nodes, determine the clustering parameters and cluster centers using the firefly algorithm, and output personalized data after clustering the independent data. Step S5. Convert the personalized data into a low-dimensional representation to obtain a personalized tag vector. Obtain a task vector through similarity aggregation. Concatenate the personalized tag vector and the task vector. Perform an inner product operation on the concatenated vector. Input the inner product result into the activation function and output the probability of the user executing the task.

[0012] Furthermore, step S1 includes: Step S11. Obtain JSON format communication logs from the business system using a lightweight SDK, and publish them to a pre-defined Topic in the message queue. The communication logs include: user ID, timestamp, task behavior, session ID, and message content. Subscribe to the Topic to obtain data streams, and build a multi-source database after pre-cleaning. Step S12. Associate the multi-source log messages within the time window using a Transformer network, extract structured information from each data record, including time, behavior, and task action, and construct a triple containing task entity, association relationship, and user attributes.

[0013] Furthermore, step S2 includes: Step S21. Construct a dynamic directed acyclic graph from all triples of data pointing to the same task target by users. Extract the task entities as nodes of the graph network, define user attributes as node attributes, and use the association relationship as directed edges connecting the nodes. Use directed GNN or temporal GNN for message passing, and update the graph structure representation by aggregating parent task and child task information. Step S22. Pool the task representations of all nodes, compress the graph size using the pooling layer, extract task-related user features using the MFEM multi-view feature extraction and fusion model, input the user features into the cross network, and output the feature vector.

[0014] Furthermore, step S3 includes: Step S31. Perform attention-weighted fusion of user feature vectors and task entity embedding vectors in the directed acyclic graph to generate cross features. Use feature hashing to map the feature vector after cross feature fusion to a high-dimensional space and project it onto a dense vector to obtain the latent vector. Step S32. Input the latent vector sequence into the TGCL time series graph contrastive learning model. Through contrastive learning and dynamic graph topology, the latent vector nodes are connected to the graph network nodes in chronological order, and the time series graph is output.

[0015] TGCL (Temporal Graph Contrastive Learning) is a self-supervised learning method that learns by improving the representation of similar samples and reducing the representation of dissimilar samples, thereby classifying nodes that meet certain conditions within the temporal graph.

[0016] Furthermore, step S4 includes: Step S41. The GCN encoder learns the temporal graph representation. A graph convolutional network is used to aggregate the adjacent information of each hop, capture homogeneous nodes, set the GCN encoder learning temporal graph, add similarity constraints between the node feature and the task vector when updating the node feature, aggregate the user's task attributes, behavior frequency and identity tags in the homogeneous nodes, and use keyword extraction or frequent item mining algorithms to filter task-related data. Step S42. The variable step size adaptive firefly algorithm is adopted. The position code of the firefly is used as the cluster center, and the inverse of the intra-cluster distance is used as the brightness function. The clustering quality under different K values ​​is evaluated, and the optimal K value and cluster center are output. K-means clustering is performed on the user according to the optimal K value and cluster center. The clustering result is used as the user's personalized data.

[0017] Furthermore, step S5 includes: Step S51. Aggregate the personal tag vectors, calculate the cosine similarity between the current user's personal tag vector and the personal tag vectors of all users in the set task neighborhood, select adjacent vectors according to the similarity, arrange them to form a matrix, input the matrix into a one-dimensional convolutional neural network for convolution and pooling, and output the task vector; Step S52. Concatenate the personality tag vector and the task vector, perform non-linear transformation and dimension alignment through a fully connected layer, and then perform element-wise inner product. The inner product result yields the interaction vector. The interaction vector is input into the Sigmoid activation function, which outputs the probability that the user will execute the set task.

[0018] Compared with the prior art, the beneficial effects achieved by the present invention are: 1. This invention can acquire customer communication logs and proactive communication data based on publish-subscribe, introduce a directed acyclic graph to simulate message passing, extract user features related to task objectives, and transform them into vector representations through knowledge graph embedding. It can dynamically extract contextual features, ensure the timeliness and completeness of data, improve the accuracy of user profiles, realize real-time and proactive user profile model construction, and enhance the personalized experience of customers.

[0019] 2. This invention can utilize the TGCL model to learn latent vectors, aggregate and capture homogeneous nodes using graph convolutional networks, determine the K value and cluster centers in the K-means clustering process, and output the clustering results as personalized data. It has higher recognition accuracy and better model generalization ability, significantly improves the recognition accuracy of user profile models, and enhances the ability of user profile models to process complex data under multi-source data and the accuracy of customer personalized tag classification.

[0020] 3. This invention compensates for information loss by converting personalized user data into a low-dimensional representation. It employs a multi-layer learner structure, inputs personalized tag vectors into a convolutional neural network aggregator, and aggregates adjacent vectors within the task's neighborhood according to similarity order to determine the prediction result for the customer performing the current task. Compared with conventional user profiling models, it has better prediction performance, achieves accurate allocation of customer resources and personalized prediction, and improves the competitiveness of recommendation algorithms. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Fig. 1 This is a schematic diagram of the structure of a customer personalized service system based on multimodal data fusion according to the present invention; Fig. 2 This is a schematic diagram illustrating the steps of a customer personalization service method based on multimodal data fusion according to the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figs. 1-2 The present invention provides a technical solution: a customer personalized service system based on multimodal data fusion, comprising: a multi-source data module, a feature vector module, a node label module, a task aggregation module, and a service prediction module; The multi-source data module is used to set the prediction task objective, publish user communication logs from various business systems to a pre-set Topic based on asynchronous publish-subscribe, obtain data streams by subscribing to the Topic, construct a multi-source database after pre-cleaning, input the multi-source data into the Transformer network of the reconstructed attention layer, extract structured information from each data record, including time, behavior and task action, and construct structured triplet data.

[0024] The multi-source data module includes: an asynchronous acquisition unit and an information extraction unit; The asynchronous acquisition unit is used to obtain JSON format communication logs from the business system through a lightweight SDK and publish them to a preset topic in the message queue. The communication logs include: user ID, timestamp, task behavior, session ID and message content. The information extraction unit is used to associate multi-source log messages within a time window through a Transformer network to construct a structured triple containing task entities, association relationships, and user attributes.

[0025] The feature vector module is used to construct a dynamic directed acyclic graph (DAG) from all triples of data pointing to the same task target. The extracted task entities are used as nodes of the graph network, user attributes are defined as node attributes, and association relationships are used as directed edges connecting the nodes. The graph neural network is used for message passing to map the task entities and association relationships of the DAG to a low-dimensional vector space. The user features related to the task target are extracted through the MFEM model and transformed into vector representations to obtain the user feature vector. The feature vector module includes: a graph neural network unit and a vector transformation unit; The graph neural network unit is used to connect user triples in chronological order to construct a directed acyclic graph, use a directed GNN or a temporal GNN for message passing, and update the graph structure representation by aggregating parent and child task information; The vector transformation unit is used to pool the task representation of all nodes, compress the graph size using the pooling layer, and extract task-related user features using the MFEM multi-view feature extraction and fusion model. The user features are input into the cross network and the output is a feature vector.

[0026] The node labeling module is used to fuse the elements in the user feature vector with the embedding vector of the corresponding task entity in the directed acyclic graph, and output a sparse vector. The sparse vector is projected onto the dense vector through a fully connected layer to obtain the latent vector. The latent vector is learned using the TGCL model, and the latent vector nodes are connected to the graph network nodes in chronological order to obtain the temporal graph. The GCN encoder learns the temporal graph representation, and the graph convolutional network is used to aggregate the adjacent information of each hop to capture homogeneous nodes. The node labeling module includes: a feature learning unit and a node aggregation unit; The feature learning unit is used to perform attention-weighted fusion of user feature vectors and embedding vectors of task entities in a directed acyclic graph to generate cross features; The node aggregation unit is used to map the feature vector after cross-feature fusion to a high-dimensional space using feature hashing, and project it onto a dense vector to obtain a latent vector.

[0027] The task aggregation module is used to extract independent data related to the user and the current task based on homogeneous nodes. The firefly algorithm is used to determine the K value and cluster center in the K-means clustering process. The clustering result is output as personalized data. The personalized data of the user is converted into a low-dimensional representation through the label embedding layer. A three-layer learner structure is used to aggregate the low-dimensional data according to the user feature vector. The loss function is reconstructed to compensate for information loss and obtain personalized label vectors. The task aggregation module includes: a network aggregation unit, a learner unit, and a K-value clustering unit; The network aggregation unit is used to input the latent vector sequence into the TGCL time-series graph contrastive learning model, and output the time-series graph through contrastive learning and dynamic graph topology. The learner unit is used to set the learning time sequence graph of the GCN encoder, add similarity constraints between the node feature and the task vector when updating the node feature, and filter task-related data; The K-value clustering unit is used to employ the variable step size adaptive firefly algorithm, which uses the firefly's position code as the cluster center and the reciprocal of the intra-cluster distance as the brightness function to evaluate the clustering quality under different K values ​​and output the optimal K value and cluster center.

[0028] The service prediction module is used to calculate the similarity of the task vector among all user personality tag vectors, take the preset number of vectors with the highest similarity as neighbor vectors, input the current user's personality tag vector into the convolutional neural network aggregator, aggregate the neighbor vectors in the task neighborhood according to the similarity order to obtain the current user's task vector, concatenate the personality tag vector and the task vector through a fully connected layer, perform an inner product operation on the concatenated vector, and output the probability value through an activation function to obtain the prediction result of the customer executing the current task.

[0029] The service prediction module includes: a feature interaction unit, a concatenated inner product unit, and a behavior prediction unit; The feature interaction unit is used to aggregate the personalized tag vectors and calculate the cosine similarity between the current user's personalized tag vector and the personalized tag vectors of all users in the set task neighborhood. The cascaded inner product unit is used to select adjacent vectors according to similarity, arrange them into a matrix, input the matrix into a one-dimensional convolutional neural network for convolution and pooling, and output the task vector. The behavior prediction unit is used to concatenate the personality tag vector and the task vector. After nonlinear transformation and dimension alignment through a fully connected layer, it performs an element-wise inner product. The inner product result yields the interaction vector. The interaction vector is input into the Sigmoid activation function, which outputs the probability that the user will perform the set task.

[0030] A method for providing personalized customer services based on multimodal data fusion includes the following steps: Step S1. Obtain user communication logs from various sources based on publish-subscribe, clean them to obtain multi-source data, input the multi-source data into the Transformer network that reconstructs the attention layer to extract structured information, and construct structured triplet data; Step S1 includes: Step S11. Obtain JSON format communication logs from the business system using a lightweight SDK, and publish them to a pre-defined Topic in the message queue. The communication logs include: user ID, timestamp, task behavior, session ID, and message content. Subscribe to the Topic to obtain data streams, and build a multi-source database after pre-cleaning. Step S12. Associate the multi-source log messages within the time window using a Transformer network, extract structured information from each data record, including time, behavior, and task action, and construct a triple containing task entity, association relationship, and user attributes.

[0031] Step S2. Connect the triples pointing to the same task objective in chronological order to construct a directed acyclic graph. Map the task entities and relationships in the directed acyclic graph to a low-dimensional vector space. Extract user features related to the task objective through the MFEM model to obtain user feature vectors. Step S2 includes: Step S21. Construct a dynamic directed acyclic graph from all triples of data pointing to the same task target by users. Extract the task entities as nodes of the graph network, define user attributes as node attributes, and use the association relationship as directed edges connecting the nodes. Use directed GNN or temporal GNN for message passing, and update the graph structure representation by aggregating parent task and child task information. Step S22. Pool the task representations of all nodes, compress the graph size using the pooling layer, extract task-related user features using the MFEM multi-view feature extraction and fusion model, input the user features into the cross network, and output the feature vector.

[0032] Step S3. The user feature vector is fused with the embedding vector of the task entity to obtain a sparse vector. The sparse vector is projected onto the dense vector through a fully connected layer to obtain a latent vector. The TGCL model learns the latent vector and connects the latent vector nodes to the graph network nodes in chronological order to obtain a temporal graph. Step S3 includes: Step S31. Perform attention-weighted fusion of user feature vectors and task entity embedding vectors in the directed acyclic graph to generate cross features. Use feature hashing to map the feature vector after cross feature fusion to a high-dimensional space and project it onto a dense vector to obtain the latent vector. Step S32. Input the latent vector sequence into the TGCL time series graph contrastive learning model. Through contrastive learning and dynamic graph topology, the latent vector nodes are connected to the graph network nodes in chronological order, and the time series graph is output.

[0033] Step S4. Use a graph convolutional network to aggregate the adjacent information of each hop in the time series graph, capture homogeneous nodes, extract independent data related to the current task based on homogeneous nodes, determine the clustering parameters and cluster centers using the firefly algorithm, and output personalized data after clustering the independent data. Step S4 includes: Step S41. The GCN encoder learns the temporal graph representation. A graph convolutional network is used to aggregate the adjacent information of each hop, capture homogeneous nodes, set the GCN encoder learning temporal graph, add similarity constraints between the node feature and the task vector when updating the node feature, aggregate the user's task attributes, behavior frequency and identity tags in the homogeneous nodes, and use keyword extraction or frequent item mining algorithms to filter task-related data. Step S42. The variable step size adaptive firefly algorithm is adopted. The position code of the firefly is used as the cluster center, and the inverse of the intra-cluster distance is used as the brightness function. The clustering quality under different K values ​​is evaluated, and the optimal K value and cluster center are output. K-means clustering is performed on the user according to the optimal K value and cluster center. The clustering result is used as the user's personalized data.

[0034] Step S5. Convert the personalized data into a low-dimensional representation to obtain a personalized tag vector. Obtain a task vector through similarity aggregation. Concatenate the personalized tag vector and the task vector. Perform an inner product operation on the concatenated vector. Input the inner product result into the activation function and output the probability of the user executing the task.

[0035] Step S5 includes: Step S51. Aggregate the personal tag vectors, calculate the cosine similarity between the current user's personal tag vector and the personal tag vectors of all users in the set task neighborhood, select adjacent vectors according to the similarity, arrange them to form a matrix, input the matrix into a one-dimensional convolutional neural network for convolution and pooling, and output the task vector; Step S52. Concatenate the personality tag vector and the task vector, perform non-linear transformation and dimension alignment through a fully connected layer, and then perform element-wise inner product. The inner product result yields the interaction vector. The interaction vector is input into the Sigmoid activation function, which outputs the probability that the user will execute the set task.

[0036] Example: Collect user multi-source communication logs, parse dialogue text, extract information using Transformer network, construct information triples, construct time series graphs, extract time series user feature vectors, obtain user latent vectors through comparative learning, cluster homogeneous nodes, enhance latent vector clustering, extract personalized user data through adaptive clustering, compensate for low-dimensionality of personalized data, aggregate task-aware vectors, cascade dual-path feature interaction, and output the predicted probability of user task execution.

[0037] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0038] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for providing personalized customer services based on multimodal data fusion, characterized in that, The method includes the following steps: Step S1. Obtain user communication logs from various sources based on publish-subscribe, clean them to obtain multi-source data, input the multi-source data into the Transformer network that reconstructs the attention layer to extract structured information, and construct structured triplet data; Step S2. Connect the triples pointing to the same task objective in chronological order to construct a directed acyclic graph. Map the task entities and relationships in the directed acyclic graph to a low-dimensional vector space. Extract user features related to the task objective through the MFEM model to obtain user feature vectors. Step S3. The user feature vector is fused with the embedding vector of the task entity to obtain a sparse vector. The sparse vector is projected onto the dense vector through a fully connected layer to obtain a latent vector. The TGCL model learns the latent vector and connects the latent vector nodes to the graph network nodes in chronological order to obtain a temporal graph. Step S4. Use a graph convolutional network to aggregate the adjacent information of each hop in the time series graph, capture homogeneous nodes, extract independent data related to the current task based on homogeneous nodes, determine the clustering parameters and cluster centers using the firefly algorithm, and output personalized data after clustering the independent data. Step S5. Convert the personalized data into a low-dimensional representation to obtain a personalized tag vector. Obtain a task vector through similarity aggregation. Concatenate the personalized tag vector and the task vector. Perform an inner product operation on the concatenated vector. Input the inner product result into the activation function and output the probability of the user executing the task.

2. The customer personalized service method based on multimodal data fusion according to claim 1, characterized in that: Step S1 includes: Step S11. Obtain JSON format communication logs from the business system using a lightweight SDK, and publish them to a pre-defined Topic in the message queue. The communication logs include: user ID, timestamp, task behavior, session ID, and message content. Subscribe to the Topic to obtain data streams, and build a multi-source database after pre-cleaning. Step S12. Associate the multi-source log messages within the time window using a Transformer network, extract structured information from each data record, including time, behavior, and task action, and construct a triple containing task entity, association relationship, and user attributes.

3. The customer personalized service method based on multimodal data fusion according to claim 2, characterized in that: Step S2 includes: Step S21. Construct a dynamic directed acyclic graph from all triples of data pointing to the same task target by users. Extract the task entities as nodes of the graph network, define user attributes as node attributes, and use the association relationship as directed edges connecting the nodes. Use directed GNN or temporal GNN for message passing, and update the graph structure representation by aggregating parent task and child task information. Step S22. Pool the task representations of all nodes, compress the graph size using the pooling layer, extract task-related user features using the MFEM multi-view feature extraction and fusion model, input the user features into the cross network, and output the feature vector.

4. The customer personalization service method based on multimodal data fusion according to claim 3, characterized in that: Step S3 includes: Step S31. Perform attention-weighted fusion of user feature vectors and task entity embedding vectors in the directed acyclic graph to generate cross features. Use feature hashing to map the feature vector after cross feature fusion to a high-dimensional space and project it onto a dense vector to obtain the latent vector. Step S32. Input the latent vector sequence into the TGCL time series graph contrastive learning model. Through contrastive learning and dynamic graph topology, connect the latent vector nodes to the graph network nodes in chronological order and output the time series graph. Step S4 includes: Step S41. The GCN encoder learns the temporal graph representation. A graph convolutional network is used to aggregate the adjacent information of each hop, capture homogeneous nodes, set the GCN encoder learning temporal graph, add similarity constraints between the node feature and the task vector when updating the node feature, aggregate the user's task attributes, behavior frequency and identity tags in the homogeneous nodes, and use keyword extraction or frequent item mining algorithms to filter task-related data. Step S42. The variable step size adaptive firefly algorithm is adopted. The position code of the firefly is used as the cluster center, and the inverse of the intra-cluster distance is used as the brightness function. The clustering quality under different K values ​​is evaluated, and the optimal K value and cluster center are output. K-means clustering is performed on the user according to the optimal K value and cluster center. The clustering result is used as the user's personalized data.

5. A customer personalization service method based on multimodal data fusion according to claim 4, characterized in that: Step S5 includes: Step S51. Aggregate the personal tag vectors, calculate the cosine similarity between the current user's personal tag vector and the personal tag vectors of all users in the set task neighborhood, select adjacent vectors according to the similarity, arrange them to form a matrix, input the matrix into a one-dimensional convolutional neural network for convolution and pooling, and output the task vector; Step S52. Concatenate the personality tag vector and the task vector, perform non-linear transformation and dimension alignment through a fully connected layer, and then perform element-wise inner product. The inner product result yields the interaction vector. The interaction vector is input into the Sigmoid activation function, which outputs the probability that the user will execute the set task.

6. A customer personalization service system based on multimodal data fusion, characterized in that, The system includes the following modules: multi-source data module, feature vector module, node label module, task aggregation module, and service prediction module; The multi-source data module is used to set the prediction task target, publish the communication logs of users from various business systems to the preset topic based on asynchronous publish-subscribe, obtain data streams by subscribing to topics, construct a multi-source database after pre-cleaning, input the multi-source data into the Transformer network of the reconstructed attention layer, extract structured information from each data record, including time, behavior and task action, and construct structured triple data; The feature vector module is used to construct a dynamic directed acyclic graph (DAG) from all triples of data pointing to the same task target. The extracted task entities are used as nodes of the graph network, user attributes are defined as node attributes, and association relationships are used as directed edges connecting the nodes. The graph neural network is used for message passing to map the task entities and association relationships of the DAG to a low-dimensional vector space. The user features related to the task target are extracted through the MFEM model and transformed into vector representations to obtain the user feature vector. The node labeling module is used to fuse the elements in the user feature vector with the embedding vector of the corresponding task entity in the directed acyclic graph, and output a sparse vector. The sparse vector is projected onto the dense vector through a fully connected layer to obtain the latent vector. The latent vector is learned using the TGCL model, and the latent vector nodes are connected to the graph network nodes in chronological order to obtain the temporal graph. The GCN encoder learns the temporal graph representation, and the graph convolutional network is used to aggregate the adjacent information of each hop to capture homogeneous nodes. The task aggregation module is used to extract independent data related to the user and the current task based on homogeneous nodes. The firefly algorithm is used to determine the K value and cluster center in the K-means clustering process. The clustering result is output as personalized data. The personalized data of the user is converted into a low-dimensional representation through the label embedding layer. A three-layer learner structure is used to aggregate the low-dimensional data according to the user feature vector. The loss function is reconstructed to compensate for information loss and obtain personalized label vectors. The service prediction module is used to calculate the similarity of the task vector among all user personality tag vectors, take the preset number of vectors with the highest similarity as neighbor vectors, input the current user's personality tag vector into the convolutional neural network aggregator, aggregate the neighbor vectors in the task neighborhood according to the similarity order to obtain the current user's task vector, concatenate the personality tag vector and the task vector through a fully connected layer, perform an inner product operation on the concatenated vector, and output the probability value through an activation function to obtain the prediction result of the customer executing the current task.

7. A customer personalization service system based on multimodal data fusion according to claim 6, characterized in that: The multi-source data module includes: an asynchronous acquisition unit and an information extraction unit; The asynchronous acquisition unit is used to obtain JSON format communication logs from the business system through a lightweight SDK and publish them to a preset topic in the message queue. The communication logs include: user ID, timestamp, task behavior, session ID and message content. The information extraction unit is used to associate multi-source log messages within a time window through a Transformer network to construct a structured triple containing task entities, association relationships, and user attributes. The feature vector module includes: a graph neural network unit and a vector transformation unit; The graph neural network unit is used to connect user triples in chronological order to construct a directed acyclic graph, use a directed GNN or a temporal GNN for message passing, and update the graph structure representation by aggregating parent and child task information; The vector transformation unit is used to pool the task representation of all nodes, compress the graph size using the pooling layer, and extract task-related user features using the MFEM multi-view feature extraction and fusion model. The user features are input into the cross network and the output is a feature vector.

8. A customer personalization service system based on multimodal data fusion according to claim 7, characterized in that: The node labeling module includes: a feature learning unit and a node aggregation unit; The feature learning unit is used to perform attention-weighted fusion of user feature vectors and embedding vectors of task entities in a directed acyclic graph to generate cross features; The node aggregation unit is used to map the feature vector after cross-feature fusion to a high-dimensional space using feature hashing, and project it onto a dense vector to obtain a latent vector.

9. A customer personalization service system based on multimodal data fusion according to claim 8, characterized in that: The task aggregation module includes: a network aggregation unit, a learner unit, and a K-value clustering unit; The network aggregation unit is used to input the latent vector sequence into the TGCL time-series graph contrastive learning model, and output the time-series graph through contrastive learning and dynamic graph topology. The learner unit is used to set the learning time sequence graph of the GCN encoder, add similarity constraints between the node feature and the task vector when updating the node feature, and filter task-related data; The K-value clustering unit is used to employ the variable step size adaptive firefly algorithm, which uses the firefly's position code as the cluster center and the reciprocal of the intra-cluster distance as the brightness function to evaluate the clustering quality under different K values ​​and output the optimal K value and cluster center.

10. A customer personalization service system based on multimodal data fusion according to claim 9, characterized in that: The service prediction module includes: a feature interaction unit, a concatenated inner product unit, and a behavior prediction unit; The feature interaction unit is used to aggregate the personalized tag vectors and calculate the cosine similarity between the current user's personalized tag vector and the personalized tag vectors of all users in the set task neighborhood. The cascaded inner product unit is used to select adjacent vectors according to similarity, arrange them into a matrix, input the matrix into a one-dimensional convolutional neural network for convolution and pooling, and output the task vector. The behavior prediction unit is used to concatenate the personality tag vector and the task vector. After nonlinear transformation and dimension alignment through a fully connected layer, it performs an element-wise inner product. The inner product result yields the interaction vector. The interaction vector is input into the Sigmoid activation function, which outputs the probability that the user will perform the set task.

Images