E-commerce data integration analysis method and system

Abstract

The invention relates to the technical field of computers, in particular to an e-commerce data integration analysis method and system, and the method comprises the steps: obtaining and associating structured business data and unstructured text data of an e-commerce platform; performing cleaning, word segmentation and word form normalization on the unstructured text data to obtain a lexical item sequence; lexical item weight features and semantic embedding features are generated based on the lexical item sequence, the lexical item weight features are jointly determined by the occurrence degree of a single text and corpus distinction degree, and the semantic embedding features are obtained by predicting context word training through head words; inputting the lexical item weight features, the semantic embedding features and the structured business data into a deep learning semantic model, outputting probability distribution and confidence of emotion or intention, and constructing physiological signal features according to the probability distribution, based on the time sequence information, the message direction information and the interaction event sequence, constructing an interaction state feature to represent a change trend of the message quantity along with time, a dialogue dominant relationship and an upgrading moment; standardizing and fusing multi-source features into a feature matrix, and establishing a perception-context matrix to realize dimension mapping from key terms to service quality; training an isolation forest algorithm tree set for each service quality dimension, randomly selecting features and segmentation points of each isolation tree, performing recursive division until a sample is isolated or meets a preset termination condition, and obtaining an expected path length; and calculating an abnormal score according to an expected path length, generating an abnormal mark and an abnormal explanation in combination with a decision path, and outputting a disposal suggestion, and using an execution result to update a perception-context matrix and a deep learning semantic model parameter. According to the method, the problems that the structured business data and the unstructured text data in the e-commerce platform are difficult to effectively associate and fuse, the service quality abnormity in the dialogue process is difficult to identify in time, and the abnormity source and the key trigger factor are difficult to explain can be solved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method and system for e-commerce data integration and analysis. Background Technology

[0002] In e-commerce online service scenarios, Quality of Service (QoS) typically requires comprehensive consideration of metrics such as responsiveness, availability, and security. These metrics are often related to the emotions and context expressed by users in text interactions such as dialogues and reviews. Therefore, relying on a single data source is insufficient for continuous monitoring and fine-grained optimization of the service process. In existing anomaly detection tasks, anomalies may manifest as fraud, abnormal transactions, abnormal behavior, or improper system use. Traditional methods (such as clustering, principal component analysis, or statistical thresholding) are prone to limitations in high-dimensional, noisy, and complex interaction conditions, and further face issues of insufficient interpretability leading to trust and compliance pressures. Meanwhile, e-commerce QoS intelligent service architectures typically include layers such as perception, network, and application. They need to coordinate text understanding, deep learning decision-making, and the simulation of emotional / physiological signals obtained from text tone mapping to support real-time, context-aware service recommendations and quality improvement. Summary of the Invention

[0003] In view of the above technical problems, this invention provides an e-commerce data integration and analysis method and system, which aims to solve the problems of difficulty in effectively associating and integrating structured business data and unstructured text data in e-commerce platforms, difficulty in timely identification of service quality anomalies during dialogue, and difficulty in explaining the source of anomalies and key triggering factors. By jointly modeling the keyword / semantic features obtained from text processing with structured information such as interaction event sequences, and introducing an isolated forest algorithm tree set for anomaly detection and a perception-context mapping mechanism for explanation, a closed-loop update of real-time anomaly detection, anomaly explanation and handling is achieved for the service quality dimension.

[0004] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0005] According to one aspect of the present invention, an e-commerce data integration and analysis method is proposed, the method comprising: Obtain structured business data and unstructured text data from an e-commerce platform, and associate the structured business data with the unstructured text data; The unstructured text data is cleaned, segmented, and lexical normalized to obtain a word sequence; Based on the term sequence, term weight features and semantic embedding features are generated. The term weight features are determined by the degree of occurrence of a term in a single text and its discriminative power in the corpus. The semantic embedding features are obtained by training a method that predicts context words with a central word. The term weight features, the semantic embedding features, and the structured business data are input into a deep learning semantic model to output the probability distribution and confidence of emotion or intent. Physiological signal features are constructed based on the probability distribution. Furthermore, interactive state features are constructed based on the time sequence information, message direction information, and interaction event sequence in the structured business data. The interactive state features are used to characterize the trend of message volume change over time, the dominant relationship in the dialogue, and the escalation moment during the dialogue process. The structured business data, physiological signal features, interaction state features, and deep learning semantic model output are standardized and fused into a feature matrix, and a perception system is established. The context matrix serves as a mapping from key terms to service quality dimensions; For each of the aforementioned service quality dimensions, an isolation forest algorithm tree set is trained. Each isolation tree in the isolation forest algorithm tree set recursively divides the input sample by randomly selecting features and randomly selecting split points within the value range of the selected features until the sample is isolated or a preset termination condition is met. The expected path length of the sample is obtained based on the split depth of the sample in each of the aforementioned isolation trees. Anomaly scores are calculated based on the expected path length. Anomaly tags and explanations are generated by combining the decision paths from the isolated forest algorithm tree set. The anomaly explanations include the corresponding service quality dimension and the triggering key terms. Action suggestions corresponding to the anomaly tags are output. The execution results of these action suggestions are used to update the perception. The context matrix and the parameters of the deep learning semantic model.

[0006] Furthermore, the structured business data includes: transaction records, access logs, interaction event sequences, service request metadata and device communication metrics, as well as physiological data collected by the terminal or wearable device, wherein the physiological data collected includes at least one or more of heart rate, body surface temperature and exercise activity metrics. The unstructured text data includes product reviews, customer service dialogues, complaint texts, descriptions of reasons for returns and exchanges, and security-related statements; when associating the structured business data with the unstructured text data, the following steps are taken: data from the same interaction link are aggregated based on session identifiers, user identifiers, or transaction identifiers, and time sequence information and message direction information are retained.

[0007] Furthermore, cleaning the unstructured text data includes: deleting account mentions, topic identifiers and link identifiers from the unstructured text data, and standardizing character encoding, capitalization, repeated symbols and noise fragments; The word segmentation and lexical normalization process includes: segmenting the text, identifying parts of speech, and restoring lexical forms to map synonymous or homographed terms to unified lexical units, and retaining the position index of the terms in the original text when generating the term sequence to support subsequent interpretation.

[0008] Furthermore, after obtaining the term sequence, frequency statistics are performed on each term in the term sequence to obtain term frequency features; a set of candidate keywords is obtained from the term sequence based on the term frequency features; The construction of the interaction state features also includes: calculating the change in the number of messages per unit time based on time sequence information to characterize the communication intensity and fluctuation between the customer and the agent, and associating the fluctuation with the service quality dimension.

[0009] Furthermore, the generation of the term weight features includes: Based on each candidate keyword in the candidate keyword set, its occurrence intensity in a single text is calculated to form the local importance of the term; The coverage of the candidate keywords in the corpus is calculated to form the global scarcity of terms; The term weight feature is obtained by combining the local importance of the term with the global scarcity of the term. When generating the semantic embedding features, the following are included: Using the current word as the center word, select its neighboring words to form a context set, train the model so that the center word can predict the words in the context set, and then iteratively update the word vector parameters.

[0010] Furthermore, the deep learning semantic model includes a pre-trained encoder based on a self-attention mechanism and a classification head. The pre-trained encoder is used to generate a contextual representation considering word order relationships. The classification head is used to map the contextual representation to scores for each emotion category or intent category, and convert the scores into the probability distribution through normalization mapping. The confidence level is determined by the maximum value of the probability distribution, the entropy value, or the inter-category difference. When the confidence level is lower than a preset condition, manual review or a backup rule engine is triggered to reduce the impact of misjudgments on the service process.

[0011] Furthermore, the constructed physiological signal features include: The emotional intensity state is determined based on the probability distribution, and the emotional intensity state is used to characterize the negative degree, urgency degree or uncertainty degree of the text feedback; Using the emotional intensity state as the driving force, trend values ​​are assigned to the physiological dimensions respectively. The physiological dimensions include at least the heart rate dimension, body surface temperature dimension, and activity state dimension. The trend values ​​are used to represent the upward trend, downward trend, or fluctuation trend of the physiological dimensions relative to the reference state. Based on the trend assignment, a physiological dimension change vector is generated, and the physiological dimension change vector is normalized and time consistency processed to eliminate different units and sampling differences. The processed physiological dimension change vector is associated with message direction information to distinguish the response patterns of the client side and the agent side, thereby obtaining the physiological signal features used for anomaly detection.

[0012] Furthermore, wherein the perception The context matrix includes: For each of the service quality dimensions, a set of key terms related to it is determined. The set of key terms includes any combination of quality defects, response delays, system failures, trust and uncertainty, assurance statements, and value and loyalty-related expressions. A keyword-dimensional association structure is formed based on the occurrence relationship of the key terms in the text. Each text entry is converted into a perceptual vector consisting of true and false tags to determine whether it contains key terms. The perceptual vector is then associated with the anomaly tags to construct a perceptual-context matrix that explains the source of anomalies. This perceptual-context matrix characterizes the impact of key terms on anomaly detection. The anomaly tags are determined by the isolated forest algorithm based on the expected path length and anomaly score. An isolated forest algorithm tree set is output to display the correspondence between key terms and anomaly tags in each of the service quality dimensions in a tree structure, thereby enabling an interpretable presentation of the anomaly detection results.

[0013] Furthermore, the desired path length includes: dividing the samples sequentially from the root node in each isolation tree of the isolation forest according to randomly selected features and randomly selected split points until the samples are separated individually or reach a preset depth, recording the number of splitting steps experienced by the samples in each isolation tree and averaging them as the desired path length. The method further includes performing dimensionality reduction visualization processing on the feature matrix to present the distribution of normal sample clusters and abnormal sample points in a low-dimensional space; The proposed solutions include at least any combination of the following: refund, replacement, compensation, work order escalation, risk interception, adjustment of content review priority, or customer service script recommendations.

[0014] According to another aspect of the present invention, an e-commerce data integration and analysis system is provided, the system comprising: The data acquisition module is used to acquire structured business data and unstructured text data from the e-commerce platform, and to associate the structured business data with the unstructured text data. The preprocessing module is used to clean, segment, and normalize the unstructured text data to obtain a word sequence. The feature generation module is used to generate term weight features and semantic embedding features based on the term sequence. The term weight features are determined by the degree of occurrence of a term in a single text and its discriminative power in the corpus. The semantic embedding features are obtained by training a method that predicts context words with a center word. The emotion recognition module is used to input the term weight features, the semantic embedding features, and the structured business data into a deep learning semantic model, output the probability distribution and confidence of emotion or intention, construct physiological signal features based on the probability distribution, and construct interaction state features based on the time sequence information, message direction information, and interaction event sequence in the structured business data. The interaction state features are used to characterize the trend of message volume change over time, the dominant relationship in the dialogue, and the escalation moment during the dialogue. The mapping module is used to standardize and fuse the structured business data, physiological signal features, interaction state features, and deep learning semantic model output into a feature matrix, and establish a perception... The context matrix serves as a mapping from key terms to service quality dimensions; An anomaly detection training module is used to train a set of isolated forest algorithm trees for each of the service quality dimensions. Each isolated tree in the isolated forest algorithm tree set randomly selects features and randomly selects split points within the value range of the selected features to recursively divide the input samples until the samples are isolated or a preset termination condition is met; and the expected path length of the sample is obtained based on the split depth of the sample in each of the isolated trees. The anomaly detection module is used to calculate an anomaly score based on the expected path length, generate anomaly tags and anomaly explanations by combining the decision paths of the isolated forest algorithm tree set, the anomaly explanations include the corresponding service quality dimensions and the triggering key terms, and output handling suggestions corresponding to the anomaly tags. The execution results of the handling suggestions are used to update the perception. The context matrix and the parameters of the deep learning semantic model.

[0015] The technical solution disclosed herein has the following beneficial effects: Compared to existing technologies, this disclosure enhances the comprehensive perception of user feedback and service processes by combining text understanding (e.g., using keyword weights and semantic representations to support the grasp of user intent / emotion) with service interaction process modeling, providing a more robust feature base for subsequent anomaly detection. By using isolated forests to identify anomalies in high-dimensional data based on randomly selected features and random segmentation, and utilizing path length to characterize the degree of anomaly, it can more effectively distinguish between normal and abnormal samples in service quality monitoring, thereby enhancing the accuracy and efficiency of monitoring. Furthermore, by constructing a matrix-based association between key terms and the service quality dimension and visually presenting the impact of keywords on anomaly identification, the interpretability of anomaly detection results can be improved, facilitating the identification of anomaly sources, triggering factors, and improvement directions, and supporting the generation of handling suggestions and a closed-loop service optimization process. Attached Figure Description

[0016] Figure 1 This is a flowchart of an e-commerce data integration and analysis method as described in the embodiments of this specification; Figure 2 This is a structural block diagram of the e-commerce data integration and analysis system in the embodiments of this specification. Detailed Implementation

[0017] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0018] Furthermore, the accompanying drawings are merely illustrative of this disclosure. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0019] This invention provides a method for integrating and analyzing e-commerce data. (Refer to...) Figure 1The diagram shown is a flowchart of an e-commerce data integration and analysis method according to an embodiment of the present invention. This method can be applied to devices such as personal computers, servers, tablets, and mobile phones. The method can be implemented by software and / or hardware, and specifically may include the following steps S101-S107: In step S101, structured business data and unstructured text data from the e-commerce platform are obtained, and the structured business data and the unstructured text data are associated.

[0020] The structured business data includes: transaction records, access logs, interaction event sequences, service request metadata and device communication metrics, as well as physiological data collected by terminals or wearable devices, including at least one or more of heart rate, body surface temperature, and exercise activity metrics; the unstructured text data includes product reviews, customer service dialogues, complaint texts, descriptions of reasons for returns and exchanges, and safety-related statements; when associating the structured business data with the unstructured text data, the following steps are taken: aggregating data from the same interaction link based on session identifiers, user identifiers, or transaction identifiers, and retaining time sequence information and message direction information.

[0021] Step S101 is specifically used to form a computable unified data view, enabling the structured records on the business side and the user expressions on the text side to correspond under the same interaction link. The structured business data is exported from the platform business system, the event tracking system, and the device-side communication module, covering transaction records, access logs, interaction event sequences, service request metadata, and device communication metrics. The interaction event sequences are used to reflect the sequential relationship between user and agent triggering behaviors, page jumps, button clicks, submissions, and cancellations in the session. The service request metadata is used to characterize the contextual conditions of a service request, including entry type, request type, processing status, and routing information. The device communication metrics are used to characterize the connection status between the terminal and the server, message delivery, and acknowledgments, thereby supporting indirect observation of latency, availability, and stability. Physiological data is collected by terminals or wearable devices and synchronized to the platform's data channel. The data can correspond to physiological or environmental inputs such as heart rate, body surface temperature, and physical activity. The sensory layer input is used to capture changes in physiological state. In scenarios lacking direct sensor data collection, physiological input can also be used as analog biological signals derived from customer communication text to characterize the impact of emotional and behavioral changes on the service experience. Unstructured text data comes from product reviews, customer service dialogues, complaint texts, descriptions of reasons for returns and exchanges, and safety-related statements. This text contains direct expressions of experience factors such as quality defects, delayed responses, system failures, trust and uncertainty, security guarantees, and value and loyalty. The association process uses session identifiers, user identifiers, or transaction identifiers as primary keys to aggregate structured records and text records generated by the same user in the same transaction or service session. When there are inconsistencies or missing identifiers from multiple sources, time proximity constraints and business rules are introduced to complete the aggregation, ensuring that text and events within the same interaction link fall into the same aggregation unit. The aggregation unit retains timestamps and forms a timeline in chronological order. Message direction information is used to distinguish the sending direction between the user side and the agent side, ensuring the restoration of dialogue rounds, response intervals, upgrade trigger points, and interaction intensity. At the same time, duplicate reports, retransmissions, and abnormal format records are deduplicated and normalized to ensure that the associated data remains consistent in field semantics, time base, and granularity.

[0022] In step S102, the unstructured text data is cleaned, segmented, and normalized to obtain a word sequence.

[0023] The cleaning of the unstructured text data includes: removing account mentions, topic identifiers, and link identifiers from the unstructured text data, and standardizing character encoding, capitalization, repeated symbols, and noisy segments. The word segmentation and lexical normalization processing includes: performing word segmentation, part-of-speech identification, and lexical restoration on the text to map synonymous or homographed terms to unified lexical units, and retaining the position index of the terms in the original text when generating the term sequence to support subsequent interpretation.

[0024] As an explanation, a usable sequence of terms is formed from unstructured text to reduce the impact of noise on subsequent feature generation and sentiment recognition. The text cleaning stage standardizes the original content such as evaluations, dialogues, and complaints, removing account mentions, topic identifiers, and link identifiers to avoid semantic interference from social symbols and external redirection information. Simultaneously, character encoding and capitalization are standardized, merging duplicate terms caused by capitalization differences, and compressing or removing duplicate symbols, redundant whitespace, abnormal separators, and meaningless fragments to make the text more morphologically stable. The word segmentation stage performs segmentation on the standardized text, obtaining a list of terms arranged in the original order. To adapt to the input format of subsequent semantic models, the same word segmentation rules as the subsequent encoder can be used to segment terms into basic word units or sub-word units. The lexical normalization stage restores the lexical forms of segmented words, mapping the same word to a unified lexical unit under different morphological changes, reducing sparsity caused by synonymous or homomorphic changes. In implementations requiring improved restoration accuracy, a more suitable restoration rule can be selected based on the part-of-speech recognition results, making the restoration of different word classes such as verbs and nouns more semantically accurate. When generating the word sequence, the position index and span information of the word in the original text are retained. The position index is used to indicate the start and end positions of each word in the character sequence or word sequence, facilitating the subsequent backtracking of the original text position of the triggering keyword and locating the corresponding sentence fragment in the anomaly interpretation stage, thus balancing the regularity of the computational input and the traceability of the interpretation output.

[0025] In step S103, term weight features and semantic embedding features are generated based on the term sequence. The term weight features are determined by the degree of occurrence of a term in a single text and its discriminative power in the corpus. The semantic embedding features are obtained by training using the center word to predict context words.

[0026] Specifically, after obtaining the term sequence, frequency statistics are performed on each term in the term sequence to obtain term frequency features; a set of candidate keywords is selected from the term sequence based on the term frequency features; the construction of the interaction state features also includes: calculating the change in the number of messages per unit time based on time sequence information to characterize the communication intensity and fluctuation between the customer and the agent, and associating the fluctuation with the service quality dimension.

[0027] Furthermore, when generating the term weight features, the process includes: calculating the occurrence intensity of each candidate keyword in a single text based on the candidate keyword set to form the term local importance; calculating the coverage of the candidate keywords in the corpus set to form the term global scarcity; combining the term local importance and the term global scarcity to obtain the term weight features; and when generating the semantic embedding features, the process includes: using the current term as the center word, selecting its neighboring terms to form a context set, training the model so that the center word can predict the terms in the context set, thereby iteratively updating the word vector parameters.

[0028] In step S103, the term sequence enters the feature construction process. Frequency statistics are used to obtain term frequency features, which are summarized according to the number of times a term appears in the same text or the same conversation text set. General stop words and terms with no information content can be excluded during the statistics, making the frequency features more focused on complaints and demands. Frequency features are used to filter candidate keyword sets. Filtering methods can include sorting by frequency and extracting high-frequency components, or combining part-of-speech recognition results to retain terms with greater semantic weight. The candidate keyword set can also be used to form a word cloud-like visualization, making the most common terms in the dialogue stand out more prominently, thus quickly indicating the main complaint signals and providing a basis for subsequent mapping and interpretation. Interaction state features are derived from time sequence information and message direction information in the same step. The message volume change curve over time is obtained by aggregating the interaction event sequence by time, and the rate of change of message volume characterizes the communication intensity and fluctuations. The message volume model can be expressed as: ; in, The curve represents the level of message volume change at time t, and N represents the total number of messages. The peaks and troughs of the curve correspond to emotional fluctuations or moments of problem relief. Combined with the sending direction, it can further reflect the dominant relationship and escalation nodes in the dialogue, thereby linking the fluctuations with the service quality dimension to form usable process features.

[0029] The term weight feature is used to transform the candidate keyword set into a numerical representation of the key terms. The weight is determined by the frequency of occurrence in the text and the corpus's discriminative power, and can be calculated using TF-IDF. ; Where t represents a term, d represents a single text entry, D represents the corpus, N represents the total number of texts in the corpus, and TF(t,d) represents the frequency of occurrence of a term within the text. The number of texts containing the term; through this combination, terms that are more prominent in the text and more discriminative in the corpus receive higher weights, resulting in a term weight vector that can be directly used as one of the inputs for subsequent semantic understanding and anomaly detection. Semantic embedding features are used to capture the semantic relationships between terms and can be trained using the Skip-gram objective of Word2Vec, predicting context words with the center word. The training objective can be represented as: ; Where T is the total number of words in the corpus. The central word at position t, Let be the context word at distance j, and c be the size of the context window. Given a central word, the probability of generating context words is calculated. By minimizing the loss function and updating the word vector parameters, an embedding representation that can characterize semantic similarity and contextual relevance is obtained for subsequent fusion modeling.

[0030] In step S104, the term weight features, the semantic embedding features, and the structured business data are input into a deep learning semantic model to output the probability distribution and confidence level of emotion or intent. Physiological signal features are constructed based on the probability distribution. Furthermore, interactive state features are constructed based on the time sequence information, message direction information, and interaction event sequence in the structured business data. The interactive state features are used to characterize the trend of message volume change over time, the dominant relationship in the dialogue, and the escalation moment during the dialogue process.

[0031] The deep learning semantic model includes a pre-trained encoder based on a self-attention mechanism and a classification head. The pre-trained encoder generates a contextual representation while considering word order relationships. The classification head maps the contextual representation to scores for each emotion category or intent category and converts the scores into the probability distribution through normalization mapping. The confidence level is determined by the maximum value, entropy value, or inter-category difference of the probability distribution. When the confidence level is lower than a preset condition, manual review or a backup rule engine is triggered to reduce the impact of misjudgments on the service process.

[0032] The construction of physiological signal features includes: determining an emotional intensity state based on the probability distribution, wherein the emotional intensity state is used to characterize the negative degree, urgency, or uncertainty of the text feedback; assigning trend values ​​to physiological dimensions using the emotional intensity state as a driving quantity, wherein the physiological dimensions include at least a heart rate dimension, a body surface temperature dimension, and an activity state dimension, and the trend values ​​are used to represent the upward trend, downward trend, or fluctuation trend of the physiological dimensions relative to a reference state; generating a physiological dimension change vector based on the trend values, and performing normalization and time consistency processing on the physiological dimension change vector to eliminate different dimensions and sampling differences; associating the processed physiological dimension change vector with message direction information to distinguish the response patterns of the customer side and the agent side, thereby obtaining the physiological signal features used for anomaly detection.

[0033] As an explanation, step S104 fuses and infers term weight features, semantic embedding features, and structured business data within the same semantic understanding chain. The semantic model employs a combination of a pre-trained encoder and a classification head. The pre-trained encoder uses a self-attention mechanism to capture long-distance dependencies at the sequence level, and the input sequence forms a context representation within the encoder. In multi-turn dialogue scenarios, the context state can be recursively updated step-by-step, and the hidden state update can be represented as... ; in, The input representation at time t, This indicates the hidden state in the previous moment. Sequence information is modeled using a self-attention mechanism. The classification head maps the context representation to a score vector of emotion category or intention category, and obtains the probability distribution through normalization mapping. The category probability can be represented in the following form: ; in, This represents the feature representation obtained by the encoder, where W and b are the classification head parameters. The probability output is for each category. The confidence level is determined by the peak value of the probability distribution or the separation between categories, reflecting the stability of the current judgment. When the confidence level is lower than the preset condition, manual review or intervention of the backup rule engine is triggered to prevent low-confidence samples from directly driving service process decisions. When constructing physiological signal features based on the probability distribution, the probability distribution is used to obtain the emotional intensity state, which is used to characterize the negative degree, urgency, or uncertainty of the text feedback, and is used as a driving force to assign trend values ​​to the heart rate dimension, body surface temperature dimension, and activity state dimension. The trend value indicates an upward trend, downward trend, or fluctuating trend relative to the reference state. The trend value is further combined into a physiological dimension change vector, and the change vector is normalized and time-consistent to eliminate the offset caused by dimensional differences and sampling inconsistencies. Then, the change vector is associated with message direction information to form physiological signal features that can distinguish the response patterns of the customer side and the agent side, so as to utilize the correspondence between sentence-level emotional output and sentence-level physiological changes in subsequent anomaly detection. Interaction state features are generated from the time sequence information, message direction information and interaction event sequence in structured business data. They are used to characterize the trend of message volume changes over time, the dominant relationship in the dialogue and the escalation moment. The dominant relationship can be obtained from the proportion and persistence of bidirectional messages in the same time slice. The escalation moment can be determined by the escalation-type triggering event or the dialogue state mutation point in the event sequence. The above interaction state features, semantic model output and physiological signal features are used as inputs for subsequent feature matrix fusion.

[0034] In step S105, the structured business data, the physiological signal features, the interaction state features, and the deep learning semantic model output are standardized and fused into a feature matrix, and a perception is established. The context matrix serves as a mapping from key terms to service quality dimensions.

[0035] Wherein, the perception The context matrix includes: determining a set of key terms associated with each of the service quality dimensions, wherein the set of key terms includes any combination of quality defects, response delays, system failures, trust and uncertainty, assurance statements, and value and loyalty-related expressions, and forming a keyword-dimension association structure based on the occurrence relationship of the key terms in the text; converting whether each text contains key terms into a perceptual vector composed of true and false labels, and associating the perceptual vector with the anomaly label to construct the perceptual-context matrix for explaining the source of anomalies, so that the perceptual-context matrix can characterize the influence of key terms on anomaly discrimination; wherein the anomaly label is determined by the isolated forest algorithm based on the expected path length and anomaly score; and outputting an isolated forest algorithm tree set to display the correspondence between key terms and anomaly labels under each of the service quality dimensions in a tree structure form, so as to achieve an interpretable presentation of anomaly detection results.

[0036] As an explanation, structured business data, physiological signal features, interaction state features, and semantic model outputs need to be fused at the same scale to form a feature matrix that can be used for subsequent anomaly detection. Features from different sources differ in dimensions, value ranges, and fluctuation amplitudes. Z-score standardization is used to align each feature dimension around its statistical center. The standardization form is as follows: ; in, The standardized value. The mean of this feature. This represents the standard deviation of the feature. After standardization, the multi-source features are concatenated or aligned at the same instance granularity to form a feature matrix. The matrix rows correspond to feedback instances or session instances, and the matrix columns correspond to various feature dimensions such as sensor data, text sentiment inference results, and interaction process metadata. This reflects the fusion approach of combining sensor-side output and natural language processing-side output to form a comprehensive dataset. After fusion, a perception-context matrix is ​​established to establish a correspondence between user perception keywords and service quality dimensions, enabling the performance and feedback under the service quality dimensions to be understood in an organized manner. Service quality dimensions can cover aspects such as quality, responsiveness, availability, security, assurance, and loyalty. Each dimension is configured with a set of related perception keywords. For example, the quality dimension is associated with quality, complaint, defective, malfunction, etc.; the responsiveness dimension is associated with refund, request, app, etc.; the availability dimension is associated with bug, issue, damage, etc.; the security dimension is associated with uncertainty, trust, etc. The assurance and loyalty dimensions can also be configured with corresponding keyword sets to identify and locate the triggering factors of user experience anomalies. The association structure between keywords and dimensions can be used to form tree-structured or matrix-based visual mappings to demonstrate the connection between specific terms and specific service dimensions, and to support subsequent explanations of anomaly sources. The generation of the perception vector uses the keyword set as the feature space, mapping whether each text contains a certain keyword to a binary element, forming a binary vector expanded along the keyword dimension; when expressing perception using binary data, the element can be written as... The binary generation process can be described using the Bernoulli distribution: ; Meanwhile, category labels can be represented in binary form and satisfy the following: ; The perception-context matrix establishes a correspondence between perception vectors and anomaly markers. Anomaly markers can be provided by the normal and abnormal results output by the isolated forest, enabling the matrix to reflect the keyword triggering and anomaly attribution of each feedback under different service quality dimensions. It also visualizes the impact of keywords such as bug, trust, and complaint on anomaly identification, enhancing the interpretability of the quality monitoring process.

[0037] In step S106, an isolated forest algorithm tree set is trained for each of the service quality dimensions. Each isolated tree in the isolated forest algorithm tree set randomly selects features and randomly selects split points within the value range of the selected features to recursively divide the input samples until the samples are isolated or a preset termination condition is met. The expected path length of the sample is obtained based on the split depth of the sample in each of the isolated trees.

[0038] The desired path length includes: dividing the samples sequentially from the root node in each isolation tree of the isolation forest according to randomly selected features and randomly selected split points until the samples are separated individually or reach a preset depth, recording the number of splitting steps experienced by the samples in each isolation tree and averaging them as the desired path length.

[0039] The proposed solutions include at least any combination of the following: refund, replacement, compensation, work order escalation, risk interception, adjustment of content review priority, or customer service script recommendations.

[0040] Subsequently, dimensionality reduction visualization processing is performed on the feature matrix to present the distribution of normal sample clusters and abnormal sample points in a low-dimensional space.

[0041] As an explanation, an ensemble of isolation forest algorithm trees is trained based on the feature matrix to identify anomalous patterns under different service quality dimensions. The isolation forest employs an ensemble approach composed of multiple isolation trees. Each isolation tree recursively partitions the input samples. The partitioning process randomly selects a feature, then randomly chooses a split point within the value range of that feature, assigning the sample to left or right child nodes. This recursion continues until a sample is isolated or the tree reaches its maximum depth. The path length from the root node to a leaf node characterizes the degree of anomalousness; a shorter path generally indicates that the sample is easier to isolate and has a higher probability of being anomalous, while a longer path is closer to a normal distribution. The expected path length is calculated based on the number of splitting steps in a single isolation tree. The path lengths of the same sample across multiple isolation trees are summed and averaged to obtain the expected path length. When scoring anomalies, the anomaly score can be obtained by normalizing the expected path length using the following formula: ; Where c(n) is used to give the average path length correction term of the binary search tree: ; The prediction result can be obtained by threshold discrimination: ; The Euler-Marcheroni constant, This is the anomaly detection threshold. After training, to visually represent the distribution of normal sample clusters and abnormal sample points in a low-dimensional space, principal component analysis can be performed on the high-dimensional features used in the isolation forest to reduce the dimensionality. The covariance matrix and projection process can be represented as follows: ; ; ; in, It is a matrix composed of the first k eigenvectors. This is a low-dimensional representation after projection. During visualization, each point corresponds to a sample, and the point's category is determined as normal or abnormal by the isolation forest. Projected features can cover embedding representations, text length, and temporal metadata, thereby showcasing the pattern differences identified by the isolation forest and enhancing interpretability. In terms of decision-making and linkage, the anomaly identification results can serve as decision inputs for the service system, using state s, action a, and the next state. This indicates the impact of a action on a service state transition. A state transition can be written as: ; Action examples can cover strategies such as refunds, exchanges, or upgrades, enabling the service process to take appropriate action and proceed to the next service state when abnormal feedback or abnormal interaction patterns are detected.

[0042] In step S107, an anomaly score is calculated based on the expected path length. An anomaly marker and an anomaly explanation are generated by combining the decision paths from the isolated forest algorithm tree set. The anomaly explanation includes the corresponding service quality dimension and the triggering key terms. A handling suggestion corresponding to the anomaly marker is output. The execution result of the handling suggestion is used to update the perception. The context matrix and the parameters of the deep learning semantic model.

[0043] The expected path length is used to indicate the ease with which samples are isolated. This is converted into anomaly scores to achieve comparable anomaly metrics. The anomaly scores can be calculated as a normalized exponential form of the path length, as described above. The process involves generating anomaly markers based on the relationship between anomaly scores and preset thresholds. These markers distinguish between normal and abnormal instances and are then linked to the corresponding service quality dimension output.

[0044] Anomaly interpretation is generated from the decision paths of the isolated forest algorithm tree set. Each decision path consists of a sequence of splitting conditions the sample traverses in each isolated tree. Summarizing the path conditions from multiple trees yields a feature set that contributes more significantly to the sample. When the feature set contains binary or weighted features related to key terms, the triggering key terms can be traced back, and the corresponding service quality dimension can be located in the perception-context matrix, thus forming the interpretation result. The interpretation content simultaneously includes the service quality dimension to which the anomaly belongs and the set of triggering key terms, ensuring the interpretation is grounded in specific feedback semantics and specific service dimensions. Handling suggestions are driven by both anomaly labeling and anomaly interpretation. Suggestion types are selected within the service process action space, which covers refunds, exchanges, compensation, work order escalation, risk interception, content review priority adjustment, and customer service script recommendations. When suggestions need to be dynamically adjusted according to business status, the suggestion strategy can be iterated using reinforcement learning's action value update form. The action value function can be expressed as... ; Where s represents the current service status, a represents the candidate action, and r represents the immediate benefit brought by the action. The next state after the treatment, As a discount factor, through the The updates gradually lead to better service recovery outcomes. The execution results of the proposed actions are used to update the perception-context matrix and semantic model parameters. When updating the perception-context matrix, a correspondence is established between executed actions and their feedback results and triggering key terms and service quality dimensions, adjusting the correlation strength between key terms and dimensions according to the actual action results. When updating the semantic model parameters, the review results or business loop results obtained after the actions are injected back into the training samples, making the distribution of sentiment or intent output more stable across similar statements. Confidence is used to constrain the impact of low-reliability outputs on the action strategy; its uncertainty can be measured by the entropy of the output probability distribution, defined as: ; in, Let C be the output probability of the i-th class, and C be the number of classes. A lower entropy value indicates a more concentrated distribution and more certain judgment, while a higher entropy value indicates a more dispersed distribution and less certain judgment. When the uncertainty reaches the preset condition, the handling suggestion is to enter the manual review or the backup rule path, thereby reducing the impact of misjudgment on the service process, and the review result is incorporated into the aforementioned update mechanism to continuously improve the effectiveness of anomaly detection and interpretation.

[0045] Based on the same content, such as Figure 2 As shown, an e-commerce data integration and analysis system is provided, the system comprising: The acquisition module 201 is used to acquire structured business data and unstructured text data from the e-commerce platform, and associate the structured business data with the unstructured text data. Preprocessing module 202 is used to clean, segment and normalize the unstructured text data to obtain a word sequence; The feature generation module 203 is used to generate term weight features and semantic embedding features based on the term sequence. The term weight features are determined by the degree of occurrence of a term in a single text and its distinguishability in the corpus. The semantic embedding features are obtained by training a method that predicts context words with a center word. The emotion recognition module 204 is used to input the term weight features, the semantic embedding features and the structured business data into a deep learning semantic model, output the probability distribution and confidence of emotion or intention, construct physiological signal features based on the probability distribution, and construct interaction state features based on the time sequence information, message direction information and interaction event sequence in the structured business data. The interaction state features are used to characterize the trend of message volume change over time, the dominant relationship in the dialogue and the escalation moment during the dialogue. The mapping module 205 is used to standardize and fuse the structured business data, the physiological signal features, the interaction state features, and the output of the deep learning semantic model into a feature matrix, and establish a perception... The context matrix serves as a mapping from key terms to service quality dimensions; Anomaly detection training module 206 is used to train a set of isolated forest algorithm trees for each of the service quality dimensions. Each isolated tree in the isolated forest algorithm tree set randomly selects features and randomly selects split points within the value range of the selected features to recursively divide the input samples until the samples are isolated or a preset termination condition is met; and the expected path length of the sample is obtained based on the split depth of the sample in each of the isolated trees. The anomaly detection module 207 is used to calculate an anomaly score based on the expected path length, generate an anomaly label and an anomaly explanation by combining the decision path of the isolated forest algorithm tree set, the anomaly explanation including the corresponding service quality dimension and the triggering key terms, and output handling suggestions corresponding to the anomaly label. The execution result of the handling suggestions is used to update the perception. The context matrix and the parameters of the deep learning semantic model.

[0046] In this embodiment, the system combines text understanding (e.g., using keyword weights and semantic representations to support the grasp of user intent / emotion) with service interaction process modeling, thereby enhancing the comprehensive perception of user feedback and service processes and providing a more robust feature base for subsequent anomaly detection. By using isolated forests to identify anomalies in high-dimensional data based on randomly selected features and random segmentation, and utilizing path length to characterize the degree of anomaly, the system can more effectively distinguish between normal and abnormal samples in service quality monitoring, thus enhancing the accuracy and efficiency of monitoring. Simultaneously, by constructing a matrix-based association between key terms and the service quality dimension and visually presenting the impact of keywords on anomaly identification, the interpretability of anomaly detection results can be improved, facilitating the identification of anomaly sources, triggering factors, and improvement directions, and supporting the generation of handling suggestions and a closed-loop service optimization mechanism.

[0047] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0048] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for integrating and analyzing e-commerce data, characterized in that, The method includes: Obtain structured business data and unstructured text data from an e-commerce platform, and associate the structured business data with the unstructured text data; The unstructured text data is cleaned, segmented, and lexical normalized to obtain a word sequence; Based on the term sequence, term weight features and semantic embedding features are generated. The term weight features are determined by the degree of occurrence of a term in a single text and its discriminative power in the corpus. The semantic embedding features are obtained by training a method that predicts context words with a central word. The term weight features, the semantic embedding features, and the structured business data are input into a deep learning semantic model to output the probability distribution and confidence of emotion or intent. Physiological signal features are constructed based on the probability distribution. Furthermore, interactive state features are constructed based on the time sequence information, message direction information, and interaction event sequence in the structured business data. The interactive state features are used to characterize the trend of message volume change over time, the dominant relationship in the dialogue, and the escalation moment during the dialogue process. The structured business data, physiological signal features, interaction state features, and deep learning semantic model output are standardized and fused into a feature matrix, and a perception system is established. The context matrix serves as a mapping from key terms to service quality dimensions; For each of the aforementioned service quality dimensions, an isolation forest algorithm tree set is trained. Each isolation tree in the isolation forest algorithm tree set recursively divides the input sample by randomly selecting features and randomly selecting split points within the value range of the selected features until the sample is isolated or a preset termination condition is met. The expected path length of the sample is obtained based on the split depth of the sample in each of the aforementioned isolation trees. Anomaly scores are calculated based on the expected path length. Anomaly tags and explanations are generated by combining the decision paths from the isolated forest algorithm tree set. The anomaly explanations include the corresponding service quality dimension and the triggering key terms. Action suggestions corresponding to the anomaly tags are output. The execution results of these action suggestions are used to update the perception. The context matrix and the parameters of the deep learning semantic model.

2. The e-commerce data integration and analysis method according to claim 1, characterized in that, The structured business data includes: transaction records, access logs, interaction event sequences, service request metadata and device communication indicators, as well as physiological data collected by the terminal or wearable device. The physiological data collected includes at least one or more of the following: heart rate, body surface temperature and exercise activity. The unstructured text data includes product reviews, customer service dialogues, complaint texts, descriptions of reasons for returns and exchanges, and security-related statements; when associating the structured business data with the unstructured text data, the following steps are taken: data from the same interaction link are aggregated based on session identifiers, user identifiers, or transaction identifiers, and time sequence information and message direction information are retained.

3. The e-commerce data integration and analysis method according to claim 1, characterized in that, Cleaning the unstructured text data includes: deleting account mentions, topic identifiers and link identifiers from the unstructured text data, and standardizing character encoding, capitalization, repeated symbols and noise fragments; The word segmentation and lexical normalization process includes: segmenting the text, identifying parts of speech, and restoring lexical forms to map synonymous or homographed terms to unified lexical units, and retaining the position index of the terms in the original text when generating the term sequence to support subsequent interpretation.

4. The e-commerce data integration and analysis method according to claim 1, characterized in that, After obtaining the term sequence, frequency statistics are performed on each term in the term sequence to obtain term frequency features; A set of candidate keywords is obtained by filtering from the term sequence based on the term frequency features; The construction of the interaction state features also includes: calculating the change in the number of messages per unit time based on time sequence information to characterize the communication intensity and fluctuation between the customer and the agent, and associating the fluctuation with the service quality dimension.

5. The e-commerce data integration and analysis method according to claim 4, characterized in that, When generating the term weight features, the following are included: Based on each candidate keyword in the candidate keyword set, its occurrence intensity in a single text is calculated to form the local importance of the term; The coverage of the candidate keywords in the corpus is calculated to form the global scarcity of terms; The term weight feature is obtained by combining the local importance of the term with the global scarcity of the term. When generating the semantic embedding features, the following are included: Using the current word as the center word, select its neighboring words to form a context set, train the model so that the center word can predict the words in the context set, and then iteratively update the word vector parameters.

6. The e-commerce data integration and analysis method according to claim 1, characterized in that, The deep learning semantic model includes a pre-trained encoder based on a self-attention mechanism and a classification head. The pre-trained encoder is used to generate a contextual representation while considering word order relationships. The classification head is used to map the contextual representation to scores for each emotion category or intent category, and convert the scores into the probability distribution through normalization mapping. The confidence level is determined by the maximum value of the probability distribution, the entropy value, or the inter-category difference. When the confidence level is lower than a preset condition, manual review or a backup rule engine is triggered to reduce the impact of misjudgments on the service process.

7. The e-commerce data integration and analysis method according to claim 1, characterized in that, The constructed physiological signal features include: The emotional intensity state is determined based on the probability distribution, and the emotional intensity state is used to characterize the negative degree, urgency degree or uncertainty degree of the text feedback; Using the emotional intensity state as the driving force, trend values ​​are assigned to the physiological dimensions respectively. The physiological dimensions include at least the heart rate dimension, body surface temperature dimension, and activity state dimension. The trend values ​​are used to represent the upward trend, downward trend, or fluctuation trend of the physiological dimensions relative to the reference state. Based on the trend assignment, a physiological dimension change vector is generated, and the physiological dimension change vector is normalized and time consistency processed to eliminate different units and sampling differences. The processed physiological dimension change vector is associated with message direction information to distinguish the response patterns of the client side and the agent side, thereby obtaining the physiological signal features used for anomaly detection.

8. The e-commerce data integration and analysis method according to claim 1, characterized in that, in, The perception The context matrix includes: For each of the service quality dimensions, a set of key terms related to it is determined. The set of key terms includes any combination of quality defects, response delays, system failures, trust and uncertainty, assurance statements, and value and loyalty-related expressions. A keyword-dimensional association structure is formed based on the occurrence relationship of the key terms in the text. Each text entry is converted into a perceptual vector consisting of true and false tags to determine whether it contains key terms. The perceptual vector is then associated with the anomaly tags to construct a perceptual-context matrix that explains the source of anomalies. This perceptual-context matrix characterizes the impact of key terms on anomaly detection. The anomaly tags are determined by the isolated forest algorithm based on the expected path length and anomaly score. An isolated forest algorithm tree set is output to display the correspondence between key terms and anomaly tags in each of the service quality dimensions in a tree structure, thereby enabling an interpretable presentation of the anomaly detection results.

9. The e-commerce data integration and analysis method according to claim 1, characterized in that, The desired path length includes: dividing the samples sequentially from the root node in each isolation tree of the isolation forest according to randomly selected features and randomly selected split points until the samples are separated individually or reach a preset depth, recording the number of splitting steps experienced by the samples in each isolation tree and averaging them as the desired path length. The method further includes performing dimensionality reduction visualization processing on the feature matrix to present the distribution of normal sample clusters and abnormal sample points in a low-dimensional space; The proposed solutions include at least any combination of the following: refund, replacement, compensation, work order escalation, risk interception, adjustment of content review priority, or customer service script recommendations.

10. An e-commerce data integration and analysis system, characterized in that, The system includes: The data acquisition module is used to acquire structured business data and unstructured text data from the e-commerce platform, and to associate the structured business data with the unstructured text data. The preprocessing module is used to clean, segment, and normalize the unstructured text data to obtain a word sequence. The feature generation module is used to generate term weight features and semantic embedding features based on the term sequence. The term weight features are determined by the degree of occurrence of a term in a single text and its discriminative power in the corpus. The semantic embedding features are obtained by training a method that predicts context words with a center word. The emotion recognition module is used to input the term weight features, the semantic embedding features, and the structured business data into a deep learning semantic model, output the probability distribution and confidence of emotion or intention, construct physiological signal features based on the probability distribution, and construct interaction state features based on the time sequence information, message direction information, and interaction event sequence in the structured business data. The interaction state features are used to characterize the trend of message volume change over time, the dominant relationship in the dialogue, and the escalation moment during the dialogue. The mapping module is used to standardize and fuse the structured business data, physiological signal features, interaction state features, and deep learning semantic model output into a feature matrix, and establish a perception... The context matrix serves as a mapping from key terms to service quality dimensions; An anomaly detection training module is used to train a set of isolated forest algorithm trees for each of the service quality dimensions. Each isolated tree in the isolated forest algorithm tree set randomly selects features and randomly selects split points within the value range of the selected features to recursively divide the input samples until the samples are isolated or a preset termination condition is met; and the expected path length of the sample is obtained based on the split depth of the sample in each of the isolated trees. The anomaly detection module is used to calculate an anomaly score based on the expected path length, generate anomaly tags and anomaly explanations by combining the decision paths of the isolated forest algorithm tree set, the anomaly explanations include the corresponding service quality dimensions and the triggering key terms, and output handling suggestions corresponding to the anomaly tags. The execution results of the handling suggestions are used to update the perception. The context matrix and the parameters of the deep learning semantic model.

Images