A dynamic construction method of a service quality evaluation system of a large language model fusion

Abstract

The application relates to a service quality evaluation system dynamic construction method of a fusion large language model, which extracts a theme, a keyword and a representative document from a user evaluation text dataset, constructs a prompt word and inputs the prompt word into a large language model to generate a SERVQUAL model five-service-dimension division candidate first-service-dimension, a candidate second-theme and a candidate third-index-item to form a service quality candidate index item list; a multi-channel heterogeneous hypergraph set is constructed by fusing a sliding window co-occurrence and an external knowledge graph; graph structure information is extracted to jointly predict a sentiment label and a sentiment score of the user evaluation text, a theme-level multi-class input factor space is constructed to calculate a final fusion weight; the candidate third-index-item is comprehensively sorted and dynamically screened to form a multi-level service quality evaluation system index scale. The application realizes accurate modeling of user perceived quality and can improve the sentiment distinguishing ability of the service quality evaluation system index.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of natural language processing and large language model application, and particularly relates to a service quality evaluation system dynamic construction method fusing a large language model. BACKGROUND

[0002] As a core link in modern service management, the construction of service quality evaluation system directly affects the response efficiency of service subjects to user demand and the optimization degree of resource allocation. The traditional service quality evaluation method represented by the SERVQUAL model constructs an evaluation framework based on five service dimensions: reliability, responsiveness, assurance, tangibility and responsiveness. In recent years, with the significant breakthroughs of large language models in natural language processing, semantic modeling and sentiment analysis, the fusion of large language model technology to enhance the traditional service quality evaluation system has become a new technology path. For example, it can effectively mine the implicit service dimensions in user feedback, identify multi-level index items, and realize systematic modeling of service attributes and user experience elements through context semantic modeling, providing a technical basis for intelligent and personalized dimension expansion and dynamic calibration of weight factors of the SERVQUAL model in a complex multi-source data environment.

[0003] However, the current service quality evaluation method based on the SERVQUAL model and its extended application system faces many challenges. First, the traditional static questionnaire evaluation system is difficult to adapt to data-driven and intelligent collaborative user service demand due to its insufficient index adaptability, limited evaluation accuracy and time lag, and other problems. It is urgent to break through the limitations of the static framework and realize dynamic construction. Second, the application of large language models in the dynamic construction of SERVQUAL model is still restricted by factors such as ambiguous user expression leading to service dimension recognition ambiguity, subjective user sentiment increasing sentiment polarity judgment difficulty, and different domain context differences requiring high demand for model generalization ability. These factors limit the existing method's accurate capture of user's real experience and real-time construction and evaluation of dynamic service quality system based on SERVQUAL model. Therefore, the construction of a service quality evaluation system construction method fusing large language model advanced reasoning ability, sentiment perception driven mechanism and multi-factor dynamic weight fusion strategy not only effectively supplements the existing traditional model, but also is a key technical foundation for accurate perception and real-time decision support of service quality in a multi-source heterogeneous data environment. SUMMARY

[0004] To solve the problems of existing service quality evaluation system construction method, such as abstract evaluation dimension, strong subjectivity of weight allocation and lack of dynamic regulation ability, the application provides a service quality evaluation system dynamic construction method combined with a large language model, which extracts keywords and representative evaluation content by using unsupervised topic modeling method based on e-commerce platform agricultural and agricultural tool user evaluation, combines structured prompt construction mechanism to generate initial index structure that can be input into large language model, and constructs multi-channel heterogeneous hypergraph neural network for semantic sentiment driven perception, enhances fine modeling of service sentiment expression, and adjusts index structure by combining multi-factor screening strategy in weight generation process to obtain service quality evaluation system index scale, improves reliability and discrimination of service evaluation index result, effectively responds to dynamic changes of multi-source heterogeneous service data, and supports real scene oriented service quality automatic evaluation.

[0005] The application is realized by the following technical solutions.

[0006] A service quality evaluation system dynamic construction method combined with a large language model, the steps are as follows:

[0007] Step one: obtain user evaluation text data set in commodity sales;

[0008] Step two: extract topics, keywords and representative documents from the user evaluation text data set, construct prompt words and input them into the large language model to generate the candidate first-level service dimensions of the SERVQUAL model five service dimension division, and the service quality candidate index item list composed of candidate secondary topics and candidate third-level index items;

[0009] Step three: take the representative documents as the test set and the remaining user evaluation text data set as the training set, and construct a multi-channel heterogeneous hypergraph set combined with sliding window co-occurrence and external knowledge graph;

[0010] Step four: input the multi-channel heterogeneous hypergraph set into the multi-channel heterogeneous hypergraph neural network model, extract graph structure information and perform joint prediction of sentiment labels and sentiment scores of user evaluation text;

[0011] Step five: according to the prediction results of sentiment labels and sentiment scores, combine user satisfaction score, user evaluation text content structure and sentiment prediction information to construct a multi-class input factor space at the topic level, calculate the final fusion weight of each topic, the secondary topic fusion weight and the normalized fusion weight on the candidate first-level service dimension;

[0012] Step six: comprehensively sort and dynamically select the candidate third-level index items, form a multi-level service quality evaluation system index scale.

[0013] Specifically, in step two, the process of extracting the theme, keywords and representative documents is as follows: each user review text is converted into a fixed-dimensional semantic embedding vector by a sentence-level multilingual pre-training model to form an overall text embedding vector matrix; then, the uniform manifold approximation and projection algorithm is used to reduce the dimension of the overall text embedding vector matrix, the local density and global structure of the semantic space are preserved, and the feature matrix after dimension reduction is obtained, the hierarchical density clustering algorithm is used to cluster the feature matrix after dimension reduction, and the theme set to which each user review text belongs is output;

[0014] The keywords of each theme are extracted through the category conditional inverse document frequency weight model;

[0015] Based on the text distribution density and clustering confidence of each theme, a representative document set is selected.

[0016] Specifically, in step two, the keywords of the theme are spliced with the corresponding representative documents, and the semantic association degree of the keywords with the five service dimensions of the SERVQUAL model is combined to generate a structured prompt input conforming to the semantic theme. For each theme, a prompt word is constructed, which includes a keyword set, a representative document set and a user satisfaction score set.

[0017] Specifically, the specific process of step three is as follows:

[0018] Each user review text in the training set and the test set is divided into a word sequence to form a semantic unit basis, and all the words constitute a vocabulary. Then, each word is mapped to a q-dimensional embedding vector based on the continuous bag-of-words model based on word embedding to obtain a word embedding matrix.

[0019] For each user review text, the word sequence is divided into a sliding window to obtain a plurality of sliding window word sets.

[0020] Based on the sliding window word set of all samples, any two words are defined and their point mutual information value is calculated, and the point mutual information threshold is set >0, the candidate hyperedge set in the user review text that meets the point mutual information threshold condition is selected;

[0021] An external knowledge graph (ConceptNet) is introduced as a second channel of the heterogeneous hypergraph. For each word, a semantic association set is defined by all connected words that have semantic edge relationships with it in ConceptNet retrieval. Then, for each word, a hyperedge set is constructed consisting of all associated semantic adjacent words.

[0022] Then the sliding window co-occurrence is uniformly fused with the hyperedge set of the two semantic channels of the external knowledge graph to form a hyperedge set of a multi-channel heterogeneous hypergraph, and a triple of the multi-channel heterogeneous hypergraph for each user review text is constructed to finally form a multi-channel heterogeneous hypergraph set constructed by all user review texts.

[0023] Specifically, in step four, the processing procedure of the multi-channel heterogeneous hypergraph neural network model includes:

[0024] First, the graph structure of the user review text is decoupled according to different semantic channels to form H channel graph structure information, and the feature embedding of each user review text under each channel is constructed, and the node feature sub-matrix is shared in all channels; for each feature embedding, a multi-layer hypergraph convolution is used to embed and learn the node features, and the structure information of adjacent nodes of different scales is aggregated layer by layer to obtain an embedding matrix with local-global semantic perception;

[0025] After L layers of multi-layer hypergraph convolution, the final node structure embedding is obtained, and after dimension alignment of the node structure embedding of all channels, a channel attention is introduced to generate a fused node embedding, which is input into a convolutional neural network, local features are extracted through local convolution, and global pooling is performed to obtain a unified graph-level structure representation, and then joint learning is performed to predict the results of sentiment labels and sentiment scores.

[0026] Specifically, the multi-class input factor space in step five includes eight index factors such as word frequency factor, sentiment average factor, document coverage factor, satisfaction driving factor, consistency factor, information density factor, text length factor and sentiment entropy factor.

[0027] Specifically, in step five, for each candidate third-level index item corresponding to a theme, minimum-maximum normalization processing is performed on all factor values to obtain normalized factor values, and then a set of preset weight parameters is used to linearly weight the normalized factor values to obtain initial fusion weights of each theme, and then the original fusion weights are normalized and converted into final fusion weights using a Softmax function.

[0028] The final fusion weight of each theme is transmitted to the corresponding candidate third-level index item to realize weight binding from bottom to top, and then a bottom-up weighted normalization strategy is used to complete weight aggregation to obtain normalized secondary theme fusion weights; at the same time, normalized fusion weights on the candidate first-level service dimension are obtained.

[0029] Specifically, in step six, the final fusion weight of the theme is screened by selecting a plurality of theme screening strategies, by traversing and comparing various theme screening strategies, selecting the theme set with the highest fusion weight value as the final core three-level index item set, and according to the selected three-level index item, combining the corresponding two-level theme and one-level service dimension to form the final service quality evaluation system scale.

[0030] The application further provides an electronic device comprising a memory and a processor, wherein the memory stores computer readable instructions, and the instructions are executed by the processor to realize the service quality evaluation system dynamic construction method.

[0031] The application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the service quality evaluation system dynamic construction method.

[0032] The application adopts a service quality evaluation method of fusing theme modeling, large language model understanding, multi-channel heterogeneous hypergraph sentiment modeling and dynamic weight updating mechanism, automatically identifies the core service theme in the user evaluation text, introduces a sliding window co-occurrence and external knowledge to construct a multi-source heterogeneous hypergraph structure, and simultaneously outputs sentiment label prediction and score results based on a multi-layer hypergraph neural network, thereby realizing multi-dimensional modeling of user perceived quality. Dynamic weight updating is performed in combination with multiple factor elements such as text length, frequency and sentiment consistency, a weighted fusion strategy is used to dynamically sort and screen service quality indicators, and finally a service quality evaluation index system with semantic representativeness, sentiment recognition and hierarchical adaptation capability is constructed. The method not only significantly improves the automation level and interpretability, but also provides a scientific basis and practical support for intelligent quality monitoring and dynamic optimization of service platforms. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a schematic diagram of the overall structure of the application;

[0034] Figure 2 is a schematic diagram of the multi-channel heterogeneous hypergraph modeling of the application;

[0035] Figure 3 is a data distribution diagram of the agricultural material and tool category of the e-commerce platform;

[0036] Figure 4 is a theme index item fusion weight ordering diagram;

[0037] Figure 5 is a radar chart of the distribution of the fusion weights of the five service dimensions of the final SERVQUAL service quality evaluation system scale. DETAILED DESCRIPTION

[0038] The present invention will be further described in detail below with reference to the accompanying drawings and examples.

[0039] Reference Figure 1 A dynamic construction method for a service quality evaluation system integrating a large language model is described in the following steps:

[0040] Step 1: Use crawler technology to crawl original user evaluation texts on agricultural supplies and agricultural tools from e-commerce platforms. Collect them in batches by product. Each original user evaluation text contains the user's free text evaluation and the corresponding user satisfaction score (1-5 points). Clean and pre-process the original user evaluation texts to obtain the user evaluation text dataset. The specific distribution is as follows: Figure 3 As shown, user reviews for agricultural implements include: seed drills, 7,132 reviews; sickles, 6,540 reviews; fertilizer spreaders, 8,276 reviews; and sprayers, 13,319 reviews. User reviews for agricultural supplies include: feed, 11,230 reviews; aquatic products, 9,756 reviews; fertilizers, 8,143 reviews; seeds, 10,278 reviews; and pesticides, 9,158 reviews.

[0041] Step 2: The user evaluation text dataset D={d1,d2,..,d n}, where d1, d2, .., d n Represent the 1st, 2nd, ..., nth user evaluation text respectively, and transform each user evaluation text d into i Convert to a fixed-dimensional semantic embedding vector , i is the user evaluation text index, Represents a set of real numbers, forming the overall text embedding vector matrix , where e1, e2, …, e n Represent the embedding vectors of the 1st, 2nd, ..., nth user evaluation text respectively, n represents the total number of user evaluation texts, and r represents the vector dimension of each user evaluation text after embedding. Subsequently, the unified manifold approximation projection algorithm (UMAP) is used to reduce the dimension of the overall text embedding vector matrix E, retaining the local density and global structure of the semantic space, and obtaining the feature matrix after dimensionality reduction. , where m represents the target dimension. On this basis, the hierarchical density clustering algorithm (HDBSCAN) is used to perform topic clustering on the feature matrix U after dimensionality reduction, and the topic set T = {t1, t2, ..., t K}, where K represents the number of topics finally obtained, t1, t2, ..., t K Represent the 1st, 2nd,…,Kth topics respectively.

[0042] To extract keywords for each topic, the category-conditional inverse document frequency weight model (c-TF-IDF) is constructed according to formula (1):

[0043] (1);

[0044] where f k,j represents the frequency of the jth word in the kth topic, df j represents the number of topics containing word j. The higher the obtained word weight, the more representative the word is of the topic. Then, based on the text distribution density and clustering confidence of each topic, a representative document set D k ={d k,1 , d k,2 ,…, d k,l} is selected as the semantic core sample of the kth topic, where D k represents the representative document set under the kth topic, l represents the number of representative documents selected in each topic, and d k,l represents the user evaluation text of the lth representative document in the kth topic. Finally, 95 topics are obtained, and the top five topics and their keywords and weight scores are shown in Table 1.

[0045] Table 1

[0046]

[0047] The keywords of the topic are spliced with the corresponding representative documents, and the semantic association degree with the five service dimensions of the SERVQUAL model (tangibility, reliability, assurance, responsiveness, and empathy) is combined to generate structured prompt input that conforms to the semantic topic. For each topic, a prompt word is constructed, and the specific construction process is shown in formula (2).

[0048] (2);

[0049] where P k represents the prompt word of the kth topic, W k represents the keyword set composed of the kth topic, represents the user satisfaction score set of the representative document set D k composed of the kth topic, respectively represent the user satisfaction scores of the 1st, 2nd, …, lth representative documents selected in the kth topic. The prompt word is input into the large language model to generate candidate primary service dimensions divided by the five service dimensions of the SERVQUAL model, where S ∈ {tangibility, reliability, assurance, responsiveness, empathy}, S k represents the candidate primary service dimension of the kth topic, and candidate secondary topics and candidate tertiary index items A list of candidate service quality indicators composed of Denotes the kth topic t k The secondary theme, Represents the kth topic t k The 1st, 2nd, ..., Gth secondary topic names, Denotes the kth topic t k The third-level indicator items, Represents the kth topic t k 1, 2, ..., The obtained list of candidate indicators of the three-level hierarchical structure service quality is shown in Table 2, and the results obtained from the first five topics are specifically selected.

[0050] Table 2

[0051]

[0052] Step 3: Use the representative documents as the test set and the remaining user evaluation text dataset as the training set. Each user evaluation text in the training set and the test set is divided into word sequences to form the semantic unit basis. The word sequence is represented as , where the word sequence O (i) is the set of all words in the i-th user evaluation text, represents the 1st, 2nd, ..., nth item of the i-th user evaluation text i words, n i Represents the number of words in the i-th user review text. All words constitute the vocabulary table V, and then the continuous bag of words model (Word2Vec) based on word embedding is used to map each word into a q-dimensional embedding vector to obtain the word embedding matrix , where |V| represents the total number of words in the vocabulary, x1,x2,…,x |V| Represent the word vector representation of the 1st, 2nd,…,|V|th word in the vocabulary respectively.

[0053] For each user evaluation text, define the window size as , for word sequence O (i) Perform sliding window division to obtain multiple sliding window word sets, as shown in formula (3) to formula (5):

[0054] (3);

[0055] (4);

[0056] (5);

[0057] in represents the sliding window word set of the i-th user evaluation text, They represent the 1st, 2nd, ..., Mth items in the i-th user evaluation text. (i) Sliding window word set, M (i) represents the total number of sliding windows generated in the i-th user evaluation text, represents the first sliding window set formed starting from the jth word in the i-th user evaluation text, Respectively represent the i-th user evaluation text Position word.

[0058] Based on the sliding window word set of all samples, any two words are defined as w b ,w p , its pointwise mutual information (PMI) value reflects the closeness of their local co-occurrence in the corpus. The calculation process is shown in formula (6):

[0059] (6);

[0060] (7);

[0061] in, For word pairs (w b ,w p ), Represents word pair (w b ,w p ) appears in all sliding windows, and Respectively represent w b ,w p Total number of occurrences, T win Indicates the total number of sliding windows generated in the entire corpus, and n represents the total number of user evaluation texts. In order to suppress the introduction of low-frequency redundant edges, the point mutual information threshold is set >0, the set of candidate hyperedges that meet the point mutual information threshold condition in the i-th user evaluation text is defined as shown in formula (8):

[0062] (8);

[0063] in, Represents the point mutual information hyperedge set constructed by the i-th user evaluation text, that is, the set of all word windows that meet the high co-occurrence strength requirements as the hyperedge in the hypergraph, reflecting the co-occurrence structural relationship between words.

[0064] In order to further model high-order semantic relationships, an external knowledge graph (ConceptNet) is introduced as the second channel of the heterogeneous hypergraph. the semantic association set defined by all the connected words that have semantic edge relations with it in ConceptNet retrieval, as shown in equation (9):

[0065] (9);

[0066] wherein represents the i-th word in the i-th user review text, the set of semantic adjacent words associated with it in the external knowledge graph, represents the tail word connected with , and represents that there is a semantic edge from to in the external knowledge graph, represents the semantic type of this edge, and ConceptNet is the external knowledge graph used. Then, for each word , a hyperedge set composed of itself and all associated semantic adjacent words is constructed, as shown in equation (10):

[0067] (10);

[0068] wherein represents the hyperedge set constructed by the semantic relations of the i-th user review text obtained through the external knowledge graph, represents that the current word and all its semantic adjacent words together form a hyperedge. Then, the sliding window co-occurrence mechanism is unified with the hyperedge sets of the two semantic channels of the external knowledge graph to form a multi-channel heterogeneous hypergraph, and the hyperedge set of the final multi-channel heterogeneous hypergraph is defined, as shown in equation (11):

[0069] (11);

[0070] wherein represents the hyperedge set of the multi-channel heterogeneous hypergraph of the i-th user review text, C represents the total number of channels of the multi-channel heterogeneous hypergraph, represents the hyperedge set from the c-th channel of the i-th user review text. Then, the triplets of the i-th user review text for the multi-channel heterogeneous hypergraph can be constructed, as shown in equation (12):

[0071] (12);

[0072] (13);

[0073] wherein, represents the multi-channel heterogeneous hypergraph of the i-th user review text, i=[1,2,…,n],​ represents the node feature submatrix obtained by embedding the words used in the i-th user evaluation text in the word embedding matrix X, |V (i) | represents the number of nodes in the i-th user evaluation text, u is the embedding dimension, S (i) Represents the graph structure of the i-th user evaluation text, represents the hyperedge connection matrix of the i-th user evaluation text, represents the user satisfaction score of the i-th user evaluation text, which ultimately constitutes a multi-channel heterogeneous hypergraph set constructed from all user evaluation texts .

[0074] Step 4: Establish a multi-channel heterogeneous hypergraph neural network model that integrates multi-layer hypergraph convolution and multi-scale feature extraction capabilities, such as Figure 2 As shown. First, the graph structure Decouple according to different semantic channels to form H channel graph structure information ,in Represents the graph structure information of the i-th graph structure under the h-th channel. The feature embedding of each user evaluation text is constructed by formula (14):

[0075] (14);

[0076] in Represents the feature embedding of the i-th user evaluation text under the h-th channel, the node feature submatrix Shared among all channels. , we use multi-layer hypergraph convolution as shown in formula (15) to embed node features and aggregate the structural information of adjacent nodes of different scales layer by layer to obtain the local-global semantically aware embedding matrix.

[0077] (15);

[0078] in represents the node embedding matrix of the i-th user evaluation text in the l+1-th hypergraph convolutional layer of the h-th channel, represents the node embedding matrix of the i-th user evaluation text in the h-th channel and the l-th layer of the hypergraph convolutional layer, represents the node degree matrix, represents the square root inverse of the node degree matrix, represents the vertex-hyperedge incidence matrix of the h-th channel, express The transpose of represents the hyperedge weight matrix of the h-th channel, represents the hyperedge degree matrix, represents the inverse of the hyperedge degree matrix, represents the feature mapping weight from the lth layer of hypergraph convolution layer to the l+1th layer of hypergraph convolution layer of the hth channel, represents a non-linear activation function.

[0079] The i-th graph structure After passing L layers through multi-layer hypergraph convolution under the h-th channel, the final node structure embedding is , embedding the node structure of all channels After dimension alignment, channel attention is introduced to generate fused node embedding, as shown in formula (16)-formula (17).

[0080] (16);

[0081] (17);

[0082] in represents the fused node embedding of the i-th graph structure, represents the attention weight of the h-th channel, represents the attention score of each channel, for The transpose of is the i-th graph structure in the The final node structure embedding obtained under the channel is represents the learnable parameter matrix in channel attention, Indicates the average pooling of node features. Embed the fusion node into Z (i) Input the convolutional neural network, extract local features through local convolution, and perform global pooling to obtain a unified graph-level structure representation f (i) Then, according to formula (18) and formula (19), joint learning is performed to predict the sentiment label and sentiment score.

[0083] (18);

[0084] (19);

[0085] in represents the predicted sentiment label of the i-th user evaluation text, represents the weight matrix of the classification task, represents the classification bias vector, represents the predicted sentiment score of the i-th user review text, represents the authority matrix of the regression task, denotes the regression bias vector, and Softmax is the activation function. The accuracy of the multi-channel heterogeneous hypergraph neural network model constructed by the application on the representative document test set reaches 78.11%, and the weighted precision, recall and F1 value are 94.04%, 78.11% and 84.24% respectively.

[0086] Step five: according to the prediction results of the emotional label and the emotional score, the application combines the user satisfaction score, the user evaluation text content structure and the emotional prediction information to construct a multi-class input factor space at the theme level, which specifically includes eight categories of index factors as shown in Table 3.

[0087] Table 3

[0088]

[0089] For each theme t k , the corresponding candidate three-level index item , the minimum-maximum normalization processing is performed on all factor values to obtain normalized factor values, as shown in formula (20):

[0090] (20);

[0091] (21);

[0092] Wherein denotes the vth normalized factor value of the kth theme t k , denotes the vth factor value of the kth theme t k , denotes the numerical set of the vth factor on K themes, denotes the vth factor value on the 1st, 2nd, …, Kth theme respectively. Then a set of preset weight parameters are used to linearly weight each normalized factor value, and the specific process is shown in formula (22):

[0093] (22);

[0094] (23);

[0095] Wherein denotes the initial fusion weight of the kth theme t k , and m denotes the total number of factors participating in weighted fusion, denotes the weight coefficient of the vth factor. Then the original fusion weight is normalized, and the Softmax function is used to convert it into the final fusion weight, as shown in formula (24).

[0096] (twenty four);

[0097] in Denotes the kth topic t k The final fusion weight of , e represents the base of the natural logarithm.

[0098] The final fusion weight of each topic Passed to its corresponding candidate third-level indicator item , realize weight downward binding, and then adopt the bottom-up weighted normalization strategy to complete weight aggregation, and obtain the normalized secondary topic fusion weight through formula (25)-formula (26).

[0099] (25);

[0100] (26);

[0101] in Denotes the kth topic t k The fusion weight of the g-th candidate secondary topic, represents the set of candidate third-level indicator items for the g-th candidate second-level topic, Indicates the A set of candidate third-level indicator items for candidate second-level topics, Denotes the kth topic t k The second-level candidate topic candidate third-level indicator items, G represents the set of all candidate second-level topics, Denotes the kth topic t k Next The fusion weight of candidate secondary topics, Denotes the kth topic t k The normalized fusion weight of the g-th candidate secondary topic is obtained. At the same time, the normalized fusion weight of the candidate primary service dimension is obtained according to formula (27).

[0102] (27);

[0103] in Denotes the kth topic t k In the candidate level service dimension S k The normalized fusion weight on represents its semantic importance contribution, S k represents the kth candidate first-level service dimension, G k Indicates that it belongs to S k The candidate secondary topic set, Indicates the Candidate first-level service dimensions, Indicates belonging A set of candidate secondary topics under a dimension.

[0104] Step 6: Candidate third-level indicators For comprehensive sorting and dynamic screening, the present invention designs three topic screening strategies to eliminate redundant topics with low fusion weight contribution from candidates, as shown in Table 4:

[0105] Table 4

[0106]

[0107] The three topic screening strategies shown in Table 4 screen the final fusion weights of the topics according to formulas (28) to (30).

[0108] (28);

[0109] (29);

[0110] (30);

[0111] in Indicates that the first The filtered topic set consists of topics. Represents the topic set retained through dynamic cumulative screening, Represents the topic set with a fusion weight higher than the overall average. The present invention traverses and compares the above three topic screening strategies and selects the topic set with the highest fusion weight as the final core three-level indicator item set, as shown in formula (31).

[0112] (31);

[0113] in represents the topic set with the highest weight fusion value, The core three-level indicator set finally retained is used as the core content of constructing the service quality evaluation system scale. The final fusion weight is as follows Figure 4 As shown. And according to the selected three-level indicators, combined with the corresponding second-level themes and first-level service dimensions, the final service quality evaluation system scale is formed. The final system results are shown in Table 5, which shows the first 10 final service quality evaluation system indicators.

[0114] Table 5

[0115]

[0116] Figure 5The application is demonstrated on the basis of user evaluation data set of agricultural material and tool commodity category in an e-commerce platform, and the normalized weight distribution of five first-level service dimensions (tangibility, assurance, reliability, responsiveness and empathy) in the service quality evaluation system finally obtained through the fusion of large language model semantic analysis, multi-channel heterogeneous supergraph modeling and multi-factor weight updating strategy. The results show that the weight of the assurance dimension reaches the highest value of 0.26, indicating that the service content such as reliable delivery, quality assurance and after-sales commitment in the user evaluation is most concerned; the weights of tangibility and responsiveness dimensions reach 0.22 and 0.17 respectively, reflecting that users have high attention to the perceptible service attributes such as product appearance, packaging experience and customer service response speed; the weights of reliability and empathy dimensions are relatively low, being 0.19 and 0.15 respectively, reflecting that some users pay attention to service stability and humanistic care.

[0117] The above results fully verify the effectiveness of the SERVQUAL service quality evaluation system proposed by the application, which fuses the understanding ability of large language model, the modeling ability of graph structure sentiment perception and the multi-factor dynamic regulation mechanism, effectively breaks through the limitations of traditional service quality evaluation methods based on questionnaires and static dimension design, and shows good migratability and practicability in e-commerce scenarios. By introducing structured evaluation index scale with strong interpretability, wide semantic coverage and dynamic adjustable index, the application provides reliable technical support for intelligent service quality evaluation, operation and maintenance optimization and strategy decision.

[0118] Another embodiment of the application provides an electronic device comprising a memory and a processor, wherein the memory stores computer readable instructions, and the instructions are executed by the processor to implement the service quality evaluation system dynamic construction method.

[0119] Still another embodiment of the application provides a computer readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the service quality evaluation system dynamic construction method.

[0120] The above description only expresses the preferred embodiments of the application, and does not limit the application in other forms. Any skilled person in the art can modify or modify the above disclosed content to equivalent embodiments. However, any simple modification, equivalent change and modification made on the basis of the technical essence of the application to the above embodiments still belongs to the protection scope of the technical solution of the application.

Claims

1. A method for dynamically constructing a quality of service evaluation system for a large language model fusion, characterized by the steps of As follows: Step one: obtain the user evaluation text dataset when the commodity is sold; Step two: extract the theme, keyword and representative document from the user evaluation text dataset, construct the prompt word and input it into the large language model to generate the SERVQUAL model five service dimension division candidate first-level service dimension, and the service quality candidate index item list composed of candidate secondary theme and candidate third-level index item; Step three: take the representative document as the test set and the remaining user evaluation text dataset as the training set to construct the multi-channel heterogeneous hypergraph set integrating sliding window co-occurrence and external knowledge graph; Each user evaluation text in the training set and the test set is divided into a word sequence to form a semantic unit basis, and all the words constitute a vocabulary table. Then, each word is mapped to a q-dimensional embedding vector by using a continuous bag-of-words model based on word embedding to obtain a word embedding matrix; For each user evaluation text, the word sequence is divided into multiple sliding window word sets by sliding window division; Defining any two words based on the sliding window word set of all samples and calculating their point mutual information value, setting the point mutual information threshold >0, selecting a candidate hyperedge set meeting the point mutual information threshold condition in the user evaluation text An external knowledge graph is introduced as the second channel of the heterogeneous hypergraph. For each word, all the semantic association sets of the connected words that have semantic edge relationship with the word are retrieved from the external knowledge graph. Then, a hyperedge set composed of each word and all the associated semantic adjacent words is constructed for each word; Then, the hyperedge sets of the two semantic channels of sliding window co-occurrence and external knowledge graph are unified and integrated to form the hyperedge set of the multi-channel heterogeneous hypergraph, and the triples of each user evaluation text for the multi-channel heterogeneous hypergraph are constructed to finally form the multi-channel heterogeneous hypergraph set constructed by all user evaluation texts; Step four: input the multi-channel heterogeneous hypergraph set into the multi-channel heterogeneous hypergraph neural network model to extract the graph structure information and jointly predict the sentiment label and sentiment score of the user evaluation text; Step five: according to the prediction results of the sentiment label and the sentiment score, combining the user satisfaction score, the content structure of the user evaluation text and the sentiment prediction information, a multi-class input factor space at the theme level is constructed, and the final fusion weight of each theme, the secondary theme fusion weight and the normalized fusion weight on the candidate first-level service dimension are calculated; The multi-class input factor space includes eight types of index factors, i.e., word frequency factor, sentiment mean factor, document coverage factor, satisfaction driving factor, consistency factor, information density factor, text length factor and sentiment entropy factor; Step six: comprehensive sorting and dynamic screening of the candidate third-level index items are performed to obtain a core third-level index item set to form a multi-level service quality evaluation system index scale.

2. The method of claim 1, wherein the method further comprises: In step two, the process of extracting the theme, keyword and representative document is as follows: each user evaluation text is converted into a fixed-dimensional semantic embedding vector by using a sentence-level multi-language pre-training model to form an overall text embedding vector matrix; then, the uniform manifold approximation and projection algorithm is used to reduce the dimension of the overall text embedding vector matrix to preserve the local density and global structure of the semantic space, and a feature matrix after dimension reduction is obtained; the hierarchical density clustering algorithm is used to cluster the feature matrix after dimension reduction to output the theme set to which each user evaluation text belongs; The keyword of each theme is extracted by using the category conditional inverse document frequency weight model. Screening a representative document set based on the text distribution density and clustering confidence of each topic.

3. The method of claim 2, wherein the method further comprises: In step two, the keywords of the topic are spliced with the corresponding representative documents, and the semantic association degree of the keywords with the five service dimensions of the SERVQUAL model is combined to generate a structured prompt input conforming to the semantic topic. For each topic, the prompt words include the keyword set, the representative document set, and the user satisfaction score set.

4. The method of claim 1, wherein the method further comprises: The external knowledge graph in step three is ConceptNet.

5. The method of claim 1, wherein the method further comprises: In step four, the processing process of the multi-channel heterogeneous hypergraph neural network model includes: First, the graph structure of the user evaluation text is decoupled according to different semantic channels to form H channel graph structure information, and the feature embedding of each user evaluation text in each channel is constructed, and the node feature sub-matrix is shared in all channels. For each feature embedding, a multi-layer hypergraph convolution is used to embed the node features, and the structure information of adjacent nodes of different scales is aggregated layer by layer to obtain an embedding matrix with local-global semantic perception; After L layers of multi-layer hypergraph convolution, the final node structure embedding is obtained. After dimension alignment of the node structure embedding of all channels, a channel attention is introduced to generate a fused node embedding. The fused node embedding is input into a convolutional neural network, local features are extracted through local convolution, and global pooling is performed to obtain a unified graph-level structure representation. Subsequently, joint learning is performed to predict the sentiment label and sentiment score.

6. The method of claim 2, wherein the method further comprises: The set of representative documents is denoted as D k ={d k,1 , d k,2 ,…, d k,l} as the semantic core sample of the kth topic, where D k represents the set of representative documents under the kth topic, l represents the number of representative documents selected in each topic, and d k,l represents the user evaluation text selected as the representative document in the kth topic.

7. The method of claim 1, wherein the method further comprises: determining a service quality evaluation system based on the service quality evaluation system information; and providing the service quality evaluation system to the user equipment. In step five, for each candidate third-level indicator item corresponding to each topic, all factor values are normalized by minimum-maximum normalization to obtain normalized factor values. Then, a set of preset weight parameters is used to linearly weight the normalized factor values to obtain the initial fusion weight of each topic. Then, the original fusion weight is normalized, and the Softmax function is used to convert it to the final fusion weight. ​ The final fusion weight of each topic is transmitted to the corresponding candidate third-level indicator item to realize weight binding from bottom to top. Then, a bottom-up weighted normalization strategy is used to complete weight aggregation to obtain the normalized secondary theme fusion weight. At the same time, the normalized fusion weight on the candidate primary service dimension is obtained.

8. The method of claim 1, wherein the method further comprises: determining a service quality evaluation system based on the service quality evaluation system information; and providing the service quality evaluation system to the user equipment. In step six, a plurality of theme screening strategies are selected to screen the final fusion weight of the theme. By traversing and comparing various theme screening strategies, the theme set with the highest fusion weight value is selected as the final core third-level indicator item set. According to the selected third-level indicator item, the corresponding secondary theme and primary service dimension form the final service quality evaluation system scale. ​ 9. An electronic device comprising a memory and a processor, the memory having stored therein computer readable instructions, wherein, The instructions are executed by the processor to implement the service quality evaluation system dynamic construction method of any one of claims 1-8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the service quality evaluation system dynamic construction method of any one of claims 1-8.

Images