E-commerce big data analysis method and system based on artificial intelligence

Abstract

The invention discloses an e-commerce big data analysis method and system based on artificial intelligence, and belongs to the field of data analysis, and the analysis method comprises the following specific steps: I, obtaining various e-commerce data from various channels, preprocessing the e-commerce data of different sources and different formats, and constructing an integrated data warehouse; iI, based on the collected e-commerce data, mining behavior characteristics of the user, extracting a fine-grained behavior pattern, and constructing a personalized label of the user; iII, analyzing each piece of text data uploaded by the user, and extracting the psychological demand, risk perception and consumption intention of the user; according to the method, the workload of manual cleaning, labeling and merging is greatly reduced, the user portrait does not stay on a static label any more, an interpretable dynamic behavior mode is formed, the causal inference and real-time key factor recognition capability is improved, and platform treatment, compliance requirements and manual checking are facilitated.

Description

Technical Field

[0001] This invention relates to the field of data analysis, and in particular to an e-commerce big data analysis method and system based on artificial intelligence. Background Technology

[0002] With the rapid development of e-commerce, user behavior data, transaction data, and content data on platforms are characterized by being multi-sourced, high-dimensional, and updated in real time. From browsing, searching, and clicking to purchasing, each step a user takes can be influenced by underlying cognitive intentions and psychological factors. Simultaneously, the platform's data exhibits heterogeneity, including structured transaction records, semi-structured log information, and unstructured text content. Extracting valuable insights from massive amounts of multi-source data has become a significant challenge for e-commerce platforms in precision marketing, user retention, and sales optimization. Traditional data analysis methods mainly rely on statistical models or simple machine learning algorithms, which struggle to simultaneously capture both explicit user behaviors (such as clicks, purchases, and searches) and implicit cognitions (such as interests, preferences, consumption intentions, and psychological needs). Furthermore, key indicators in e-commerce scenarios (such as sales volume, conversion rate, and user loyalty) are influenced by multiple interacting factors, exhibiting temporal dynamics and non-linear causal relationships, making accurate causal analysis and trend prediction difficult with traditional methods.

[0003] A search revealed that Chinese patent CN117874446A discloses an AI-based e-commerce big data analysis method and system. While this invention achieves accurate and reliable processing of e-commerce product data and efficient management and allocation of products to be assigned, the workload of manual cleaning, labeling, and merging is high, and user profiles are limited to static labels, failing to form interpretable dynamic behavioral patterns. This reduces the ability to make causal inferences and identify key factors in real time, which is detrimental to platform governance, compliance requirements, and manual verification. Therefore, we propose an AI-based e-commerce big data analysis method and system. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing an artificial intelligence-based e-commerce big data analysis method and system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: An artificial intelligence-based e-commerce big data analysis method, the specific steps of which are as follows: I. Acquire various types of e-commerce data from multiple channels, preprocess e-commerce data from different sources and in different formats, and build an integrated data warehouse; II. Based on the collected data from various e-commerce platforms, we will mine user behavioral characteristics and extract fine-grained behavioral patterns to construct personalized user tags. III. Analyze the various text data uploaded by users to extract their psychological needs, risk perceptions, and consumption intentions; IV. Decouple and analyze users' explicit behaviors and implicit intentions separately to dynamically identify various factors that affect sales, conversion rates, and user loyalty; V. Based on the characteristics of changes in user consumption, construct a cognitive evolution map and analyze the potential trends in market demand. Based on the analysis results, generate actionable strategy recommendations.

[0006] As a further aspect of the present invention, the specific steps of preprocessing e-commerce data from different sources and in different formats in step I, and constructing an integrated data warehouse, are as follows: S1.1: Randomly extract multiple sets of e-commerce data from various data sources, and list the field set and field example values ​​for each source. Then calculate the similarity between the field names and example values ​​of any two data sources. If the similarity is higher than the preset threshold, the field pair is listed as an automatic mapping candidate and a field mapping candidate pair is established. Otherwise, a mapping missing report is generated and marked as "requires manual review". S1.2: Perform distribution checks on the numerical fields in each e-commerce data and identify outliers. At the same time, generate an anomaly list to be manually reviewed. After the deduplication and anomaly handling of each e-commerce data are completed, generate and record the corresponding change log. S1.3: Fill the missing values ​​in the category fields of each e-commerce data by adding a new default category, then fill the missing values ​​in the numerical fields of each e-commerce data by using linear time series interpolation, then unify the time fields in each e-commerce data, and convert all time fields into standard numerical timestamps. S1.4: Unify the units of numerical fields in e-commerce data, convert fields with different units of measurement according to the mapping table, select the z-score standardization method to standardize the numerical fields, record the parameters used for conversion for each field, select the corresponding encoding method for the category field according to the corresponding cardinality, and save the mapping table as the corresponding metadata. S1.5: Based on historical experience or manual settings of primary key merging strategies for e-commerce data from various data sources, and in accordance with the preset primary key merging strategy, align and merge records from different sources that point to the same business entity. At the same time, when the same attribute conflicts in different sources, synthesize the final value according to the weight priority strategy, while retaining source traceability information and uncertainty measurement. S1.6: Write the merged e-commerce data into the data warehouse and add corresponding metadata for each piece of e-commerce data. The metadata includes the source list, merging strategy, cleaning log, last update timestamp and quality score.

[0007] As a further aspect of the present invention, the specific calculation formula for the linear time series interpolation method described in S1.3 is as follows: In the formula, This represents the estimated value at time after interpolation; This represents the observation at the most recent known time before the missing point; This represents the observation at the most recent known time after the missing point; This represents the timestamp corresponding to the subsequent value; This represents the timestamp corresponding to the previous value; A timestamp representing the missing point.

[0008] As a further aspect of the present invention, the specific steps for constructing the user's personalized tags in step II are as follows: S2.1: Extract and preprocess user behavior data from e-commerce data from various data sources, sort all original behavior records of each user in chronological order, calculate the time gap between two adjacent records, and determine whether they are the same session or segment. If the time gap exceeds the preset idle threshold, it is considered a session boundary. Based on the identified session boundary, the session can be divided into multiple segments. Then, add the corresponding session identifier and segment number to each processed behavior record. S2.2: Calculate the basic behavioral features corresponding to session level, user level and time window level respectively, then calculate the granularity and time window boundary of each basic behavioral feature, while retaining the original distribution statistics within the time window, create an intermediate feature table, record each basic behavioral feature, and then calculate the recording time and sample size of each feature in the intermediate feature table. S2.3: Map user behavior data to corresponding event embedding sequences, count the frequency of state transitions within the same session or across sessions, and construct an empirical transition probability matrix to obtain the corresponding behavior path. Smooth and regularize the empirical transition probability matrix, and establish multiple sets of different types of input sequences. Then, select and train the corresponding sequence model according to the business objectives. S2.4: Based on the sequence model type after training, input the corresponding input sequence. Then, the sequence model outputs multiple sets of behavior vectors through forward propagation, and performs clustering or dimensionality reduction analysis on each set of behavior vectors to obtain the corresponding behavior prototype. Then, set a label set and design a corresponding mapping rule for each label. S2.5: A non-linear scoring function is used to map user behavior vectors to tag scores, and confidence and backtracking basis are calculated for each tag. If the tag score is greater than or equal to a preset threshold, a personalized tag for the user is directly generated based on the user behavior. Otherwise, the current user behavior is marked as pending manual review, and the tag timestamp and source are recorded.

[0009] As a further aspect of the present invention, the basic behavioral characteristics mentioned in S2.2 specifically include: event count, number of unique event types, average dwell time, bounce rate, number of conversion attempts, and time since the last interaction.

[0010] As a further aspect of the present invention, the specific steps for extracting the user's psychological needs, risk perception, and consumption intention in step III are as follows: S3.1: Perform language recognition and unified character encoding on various types of user text data respectively, then record the annotation language of non-target language and select translation or removal based on platform requirements, then remove HTML tags, URLs, repeated whitespace, and non-semantic noise symbols from each text data, and then replace special placeholders in user input with unified symbols. S3.2: Perform word segmentation and sub-word segmentation on the text data respectively, generate corresponding word sequence and record the original offset of the word sequence, then unify the capitalization of each word and sentence in the word sequence, remove stop words, then extract the stem of each word and sentence, and generate a normalized word sequence, original offset index, language identifier and preprocessing log for each text data. S3.3: Extract multiple sets of semantic features from each preprocessed text data through a semantic encoder, and perform batch normalization and PCA dimensionality reduction on each set of semantic features. Then, record each processed semantic feature in the feature index, and record the generator version, weight ID and normalization parameters. S3.4: A multimodal semantic discriminator is used for each semantic feature to generate confidence scores and thresholded judgment results for various labels of intent recognition, sentiment tendency and risk perception detection, while marking the key words or sentence positions of each label for recognition; S3.5: Extract entity categories from text data and record the character offset and context window of each entity. Calculate the context compatibility and string similarity between various data and the extracted entities in the platform's product knowledge base, brand library, common phrase table and external knowledge base. If the context compatibility or string similarity is higher than the preset threshold, mark the corresponding data as a candidate and generate a candidate list. S3.6: Calculate the confidence score of each entity-candidate pair, and select the entity-candidate pair with the highest confidence score as the optimal mapping. At the same time, mark entities that fail to match as "multiple candidates require manual review". Then output the final entity identifier and traceability information. Based on the text features and entity identifiers, evaluate the semantic matching degree and infer the implicit relationship pointed to by the text. Calculate the matching score of each text feature for the entity identifier, and based on the obtained matching score, generate a structured ternary expression of the corresponding user psychological needs, risk perception and consumption intention.

[0011] An artificial intelligence-based e-commerce big data analysis system includes a data acquisition and processing module, a storage management module, a behavior analysis module, a cognitive recognition module, a fusion analysis module, a product analysis module, an inference and recognition module, a trend analysis module, a graph construction module, a risk warning module, a decision recommendation module, and an interactive monitoring module. The data acquisition and processing module is used to collect e-commerce data from multiple data sources in real time and preprocess the collected e-commerce data. The storage management module utilizes a distributed storage and data lake architecture to classify, store, and manage the pre-processed e-commerce data in a unified manner. The behavior analysis module is used to extract user behavior features from various behavior information and identify user behavior patterns; The cognitive recognition module is used to analyze user comments, search terms, and consultation content to construct the user's cognitive characteristics; The fusion analysis module decouples and analyzes user motivation based on user behavior characteristics and cognitive characteristics; The product analysis module is used to analyze product attribute tags, price curves, evaluation characteristics, and competitive environment; The inference and identification module is used to infer macroeconomic factors and individual user behavior respectively, and to identify each influencing factor; The trend analysis module is used to identify changes in demand, evolution of user sentiment, and trends in various e-commerce information throughout the product lifecycle. The graph construction module establishes a cognitive evolution path graph based on changes in user cognition; The risk warning module detects potential risks based on trend analysis and inference results, while identifying market opportunities and providing early warnings. The decision recommendation module generates executable optimization strategies based on various analysis results; The interactive monitoring module is used to visualize the output of each module.

[0012] As a further aspect of the present invention, the specific steps of the fusion analysis module in decoupling the analysis of user motivation are as follows: S4.1: Based on the user's behavioral and cognitive characteristics, establish corresponding behavioral and cognitive graphs, and map user behavior and cognition to one or more nodes on the behavioral and cognitive graphs. Generate initial edge candidates for the behavioral and cognitive graphs by statistically counting co-occurrence and context window co-occurrence rate. S4.2: Filter out candidate edges whose confidence does not meet the preset threshold, and initialize weight and time label for each remaining edge to establish two adjacency lists and a node attribute table. The adjacency list includes a behavioral edge list and a semantic edge list. S4.3: Set the feature clusters of each node in the behavior graph and cognitive graph, then concatenate the multimodal features and map them to a unified dimension through a small feedforward network to generate the initial node features corresponding to each node. Then, according to the preset requirements, determine the connection edge features between each node and generate the corresponding edge features through normalization and discrete encoding. S4.4: Input the behavior graph and cognitive graph into the heterogeneous graph neural network. At the same time, the heterogeneous graph neural network extracts the adjacency structure, initial node features and edge features from each user's behavior graph and cognitive graph in parallel. Through the forward propagation algorithm, it performs the steps of collecting messages from neighboring nodes, updating node representations based on edge features and weighted aggregation, and outputs the prediction probability of the user's next behavior and the prediction of topic tags. Based on the prediction results, it generates the corresponding behavior driving components and cognitive driving components for the user and records the contribution of each driving component to the current user.

[0013] As a further aspect of the present invention, the specific steps of the inference and identification module to infer macroeconomic factors and individual user behavior, and to identify each influencing factor, are as follows: S5.1: The indicators to be monitored and the required intermediate variables from the behavioral and cognitive paths are used as the node set of the graph. The co-occurrence and sequential occurrence counts of nodes in each time window are counted. Then, the co-occurrence probability and marginal probability of each pair of nodes in each time window are calculated to obtain the corresponding candidate causal strength. If there is a candidate causal strength lower than the preset confidence threshold, the edge between the corresponding nodes is removed, and the candidate causal adjacency matrix at the corresponding time is generated. At the same time, the causal graph is constructed. S5.2: At each time point, the target is set to be jointly acted by candidate parent nodes in a linear or generalized linear manner, and the structural coefficient matrix of the corresponding time is calculated in the sliding window through a regression form containing smoothing constraints. The significance of each calculated structural coefficient is tested, and the candidate edges that pass the test are used as real-time causal edges. S5.3: Based on the newly generated structural coefficient matrix, update the weights of each connecting edge and the local propagation rules in the causal graph, then update the hidden representations of each node and edge in the causal graph according to the current node and edge features, and output the prediction and influence propagation mode in real time, while generating short-term causal propagation paths. S5.4: Collect all user driving factors, randomly select any driving factor for random perturbation, and predict the changes of the corresponding target at a preset future time point through a graph propagation model. At the same time, use Monte Carlo simulation to calculate the distribution and uncertainty of the prediction results, and output the expected causal effect and confidence interval. S5.5: Sort the expected causal effects of each candidate driving factor from high to low according to the absolute value, output a real-time list of key factors, and record the confidence level and main propagation path of each driving factor. Then set the business threshold and combine the ranking results with the threshold rules to form an automatic alarm rule.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention randomly samples from various data sources, analyzes field names and example values, calculates field similarity, and automatically generates mapping candidates. For fields that cannot be matched, it generates manual review reports, performs noise reduction, outlier handling, and missing value imputation. Under the primary key merging strategy, it aligns and merges multi-source data, synthesizes conflicting fields according to priority, retains traceability information, and finally writes them into the data warehouse. Then, it extracts user behavior records and sorts them by time, identifies session and fragment boundaries, generates multi-level behavioral features and intermediate feature tables, and constructs event embedding and transition probability matrices. Then, it generates behavioral vectors through sequence models, extracts behavioral prototypes, and assigns personalized labels to users using non-linear scoring functions. Low-confidence samples are labeled and manually reviewed. After semantic encoder and dimensionality reduction processing, a text feature index is established, and then a multimodal semantic discriminator identifies intent, emotion, and risk. Subsequently, text entities are extracted and matched with a knowledge base to form entity mappings and structured ternary relationships, used to infer user psychological needs, risk perception, and consumption intentions. Behavioral and cognitive graphs are constructed based on behavioral and cognitive features, respectively. These graphs are then processed by a heterogeneous graph neural network to learn node and edge representations, outputting behavioral predictions and cognitive driving components. Next, candidate causal strengths are statistically analyzed based on time windows to establish a dynamic graph causal model. Causal coefficients are estimated using a sliding window structural equation model, updating causal edges and propagation patterns. Random perturbations are applied to driving factors, and future impacts are predicted through graph propagation. Monte Carlo simulations are used to estimate uncertainty. Finally, key factors are ranked according to expected causal effects, and automatic early warning rules are generated by combining thresholds. This significantly reduces the workload of manual cleaning, labeling, and merging, allowing user profiles to move beyond static labels and form interpretable dynamic behavioral patterns. This improves causal inference and real-time key factor identification capabilities, benefiting platform governance, compliance requirements, and manual verification. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0016] Figure 1 This is a flowchart of an e-commerce big data analysis method based on artificial intelligence proposed in this invention. Figure 2 This is a system block diagram of an e-commerce big data analysis system based on artificial intelligence proposed in this invention. Detailed Implementation

[0017] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] Example 1 Reference Figure 1 This embodiment discloses an e-commerce big data analysis method based on artificial intelligence. The specific steps of the analysis method are as follows: We acquire various types of e-commerce data from multiple channels, preprocess e-commerce data from different sources and in different formats, and build an integrated data warehouse.

[0019] Specifically, multiple sets of e-commerce data are randomly selected from various data sources, and the field set and example values ​​for each source are listed. Then, the similarity between field names and example values ​​of any two data sources is calculated. If the similarity exceeds a pre-set threshold, the field pair is listed as an automatic mapping candidate, and a field mapping candidate pair is established. Otherwise, a mapping missing report is generated and marked as "requires manual review." Distribution checks are performed on the numerical fields in each e-commerce data set to identify outliers, and a list of anomalies awaiting manual review is generated. After deduplication and anomaly handling are completed for each e-commerce data set, a corresponding change log is generated and recorded. Missing values ​​in the classification fields of each e-commerce data set are filled using a newly added default category. Then, missing values ​​in the numerical fields of each e-commerce data set are filled using linear time series interpolation. Finally, the time fields in each e-commerce data set are unified, and all time fields are converted to standard numerical timestamps, thus unifying the e-commerce data. The system identifies the units of numerical fields and converts fields with different units of measurement using a mapping table. It then standardizes the numerical fields using the z-score standardization method, recording the conversion parameters for each field. Next, it selects the appropriate encoding method for the categorical fields based on the corresponding cardinality and saves the mapping table as corresponding metadata. Based on historical experience or manual settings, it sets primary key merging strategies for e-commerce data from various data sources. Following the preset primary key merging strategy, it aligns and merges records from different sources that point to the same business entity. Simultaneously, when the same attribute conflicts between different sources, it synthesizes the final value according to a weighted priority strategy, while retaining source traceability information and uncertainty metrics. The merged e-commerce data is then written to the data warehouse, and corresponding metadata is added to each piece of e-commerce data. This metadata includes a source list, merging strategy, cleaning logs, and the last update timestamp and quality score.

[0020] It should be further explained that the specific calculation formula for linear time series interpolation is as follows: In the formula, This represents the estimated value at time after interpolation; This represents the observation at the most recent known time before the missing point; This represents the observation at the most recent known time after the missing point; This represents the timestamp corresponding to the subsequent value; This represents the timestamp corresponding to the previous value; A timestamp representing the missing point.

[0021] Based on the collected data from various e-commerce platforms, we can mine user behavioral characteristics and extract fine-grained behavioral patterns to build personalized user tags.

[0022] Specifically, user behavior data is extracted and preprocessed from e-commerce data from various data sources. All original behavior records for each user are sorted chronologically, and the time interval between adjacent records is calculated to determine if they belong to the same session or segment. If the time interval exceeds a preset idle threshold, it is considered a session boundary. Based on the identified session boundaries, the session is further divided into multiple segments. Then, each processed behavior record is assigned a corresponding session identifier and segment number. Basic behavioral features are calculated at the session, user, and time window levels. The granularity and time window boundary of each basic behavioral feature are then calculated, while retaining the original distribution statistics within the time window. An intermediate feature table is created, recording each basic behavioral feature. The time and sample size of each feature record in the intermediate feature table are then calculated. User behavior data is mapped to corresponding event embedding sequences, and statistics are collected for the same session or across sessions. The system calculates the frequency of state transitions and constructs an empirical transition probability matrix to obtain corresponding behavioral paths. The empirical transition probability matrix is ​​then smoothed and regularized. Multiple sets of different types of input sequences are established, and corresponding sequence models are selected and trained based on business objectives. Based on the type of the trained sequence model, the corresponding input sequence is input. This sequence model then outputs multiple sets of behavioral vectors through forward propagation. Clustering or dimensionality reduction analysis is performed on each set of behavioral vectors to obtain corresponding behavioral prototypes. A label set is then set, and corresponding mapping rules are designed for each label. A non-linear scoring function maps user behavior vectors to label scores, and confidence and backtracking basis are calculated simultaneously for each label. If the label score is greater than or equal to a preset threshold, a personalized label for the user is directly generated based on the user behavior; otherwise, the current user behavior is marked as requiring manual review, and the label timestamp and source are recorded.

[0023] In this embodiment, the basic behavioral characteristics specifically include: event count, number of unique event types, average dwell time, bounce rate, number of conversion attempts, and time since the last interaction.

[0024] Analyze the text data uploaded by users to extract their psychological needs, risk perceptions, and consumption intentions.

[0025] Specifically, language recognition is performed on various types of user text data, and character encoding is standardized. Then, the annotation language for non-target languages ​​is recorded, and translation or removal is selected based on platform requirements. Next, HTML tags, URLs, duplicate whitespace, and non-semantic noise symbols are removed from each text data. Special placeholders in user input are replaced using standardized symbols. The text data is then segmented into words and sub-words, generating corresponding word sequences and recording their original offsets. The capitalization of words and phrases in the word sequences is standardized, and stop words are removed. Stemming is then performed on each word and phrase, generating a standardized word sequence, original offset index, language identifier, and preprocessing log for each text data item. A semantic encoder extracts multiple sets of semantic features from each preprocessed text data item, and batch normalization and PCA dimensionality reduction are performed on each set of semantic features. The processed semantic features are then entered into a feature index, along with the generator version, weight ID, and normalization parameters. A multimodal semantic discriminator is used to generate intent recognition and emotion recognition for each semantic feature. The system detects the confidence scores and thresholded judgment results of various labels for perception of tendency and risk. It also marks the positions of keywords or sentences for each label. Entity categories are extracted from the text data, and the character offset and context window of each entity are recorded. The system calculates the context compatibility and string similarity between various data and extracted entities in the platform's product knowledge base, brand database, common phrase table, and external knowledge base. If the context compatibility or string similarity is higher than the preset threshold, the corresponding data is marked as a candidate and a candidate list is generated. The confidence score of each entity-candidate pair is calculated, and the entity-candidate pair with the highest confidence score is selected as the optimal mapping. Entities that fail to match are marked as "multiple candidates require manual review". The final entity identifier and traceability information are then output. Based on the text features and entity identifiers, the semantic matching degree is evaluated, and the implicit relationship pointed to by the text is inferred. The matching score of each text feature with respect to the entity identifier is calculated. Based on the obtained matching scores, a structured ternary expression of the corresponding user psychological needs, risk perception, and consumption intention is generated.

[0026] We decouple and analyze users’ explicit behaviors and implicit intentions separately to dynamically identify various factors that affect sales, conversion rates, and user loyalty.

[0027] Based on the changing characteristics of user consumption, a cognitive evolution map is constructed, and the potential trends of market demand are analyzed. Based on the analysis results, actionable strategy recommendations are generated.

[0028] Example 2 Reference Figure 2This embodiment discloses an e-commerce big data analysis system based on artificial intelligence, including a data acquisition and processing module, a storage management module, a behavior analysis module, a cognitive recognition module, a fusion analysis module, a product analysis module, an inference and recognition module, a trend analysis module, a graph construction module, a risk warning module, a decision recommendation module, and an interactive monitoring module; The data acquisition and processing module is used to collect e-commerce data from multiple data sources in real time and preprocess the collected e-commerce data; the storage management module uses distributed storage and data lake architecture to classify, store and manage the preprocessed e-commerce data in a unified manner. The behavior analysis module is used to extract user behavior features from various behavioral information and identify user behavior patterns; the cognitive recognition module is used to analyze user comments, search terms, and consultation content to construct user cognitive features; the fusion analysis module decouples and analyzes user motivation based on user behavior features and cognitive features.

[0029] Specifically, based on user behavior and cognitive characteristics, corresponding behavior graphs and cognitive graphs are established, and user behavior and cognition are mapped to one or more nodes on the behavior graphs and cognitive graphs. Initial edge candidates for the behavior graphs and cognitive graphs are generated by statistically analyzing co-occurrence counts and context window co-occurrence rates. Candidate edges whose confidence does not meet a preset threshold are filtered out, and weights and time labels are initialized for each remaining edge to establish two adjacency lists and a node attribute table. The adjacency lists include a behavior edge list and a semantic edge list. Feature clusters for each node in the behavior graphs and cognitive graphs are set, and multimodal features are concatenated and mapped to a unified dimension through a small feedforward network to generate the initial nodes corresponding to each node. The system first identifies the features of the connections between nodes based on preset requirements. Then, it generates corresponding edge features through normalization and discrete encoding. The behavior graph and cognitive graph are input into the heterogeneous graph neural network. Simultaneously, the heterogeneous graph neural network extracts the adjacency structure, initial node features, and edge features from each user's behavior graph and cognitive graph in parallel. Through the forward propagation algorithm, it executes the steps of collecting messages from neighboring nodes, updating node representations based on edge features, performing weighted aggregation, and outputting the probability of the user's next behavior prediction and the prediction of topic tags. Based on the prediction results, it generates the user's corresponding behavior driving components and cognitive driving components, and records the contribution of each driving component to the current user.

[0030] The product analysis module is used to analyze product attribute tags, price curves, evaluation characteristics, and competitive environment; the inference and identification module is used to infer macro factors and individual user behavior, and identify various influencing factors.

[0031] Specifically, the monitored indicators and the necessary intermediate variables from behavioral and cognitive paths are used as the node set of the graph. The co-occurrence and sequential occurrence counts of nodes within each time window are statistically analyzed. Then, the co-occurrence probability and marginal probability of each pair of nodes within each time window are calculated to obtain the corresponding candidate causal strength. If a candidate causal strength is lower than a preset confidence threshold, the edge between the corresponding nodes is removed, and a candidate causal adjacency matrix for the corresponding time is generated. Simultaneously, a causal graph is constructed. At each time point, the target is set to be jointly acted upon by candidate parent nodes in a linear or generalized linear manner. Within a sliding window, the structural coefficient matrix for the corresponding time is calculated using a regression form with smoothing constraints. The significance of each calculated structural coefficient is tested, and the candidate edges that pass the test are used as real-time causal edges. Based on the latest generated... The structural coefficient matrix updates the weights of each connecting edge and the local propagation rules in the causal graph. Then, based on the current node and edge characteristics, it updates the hidden representations of each node and edge in the causal graph and outputs the prediction and influence propagation patterns in real time. At the same time, it generates short-term causal propagation paths, collects various user driving factors, and randomly selects any driving factor for random perturbation. Simultaneously, it predicts the changes of the corresponding target at a preset future time point through a graph propagation model. Monte Carlo simulation is used to calculate the distribution and uncertainty of the prediction results, and outputs the expected causal effect and confidence interval. The expected causal effects of each candidate driving factor are sorted from high to low absolute value, and a real-time list of key factors is output. The confidence level and main propagation path of each driving factor are recorded. Then, a business threshold is set, and the ranking results are combined with the threshold rules to form an automatic alarm rule.

[0032] The trend analysis module is used to identify changes in demand, evolution of user sentiment, and changes in e-commerce information throughout the product lifecycle; the graph construction module establishes a cognitive evolution path graph based on changes in user cognition.

[0033] The risk warning module detects potential risks based on trend analysis and inference results, while identifying market opportunities and providing early warnings; the decision recommendation module generates actionable optimization strategies based on various analysis results; and the interactive monitoring module is used to visualize the outputs of each module.

Claims

1. An e-commerce big data analysis method based on artificial intelligence, characterized in that, The specific steps of this analytical method are as follows: I. Acquire various types of e-commerce data from multiple channels, preprocess e-commerce data from different sources and in different formats, and build an integrated data warehouse; II. Based on the collected data from various e-commerce platforms, we will mine user behavioral characteristics and extract fine-grained behavioral patterns to construct personalized user tags. III. Analyze the various text data uploaded by users to extract their psychological needs, risk perceptions, and consumption intentions; IV. Decouple and analyze users' explicit behaviors and implicit intentions separately to dynamically identify various factors that affect sales, conversion rates, and user loyalty; V. Based on the characteristics of changes in user consumption, construct a cognitive evolution map and analyze the potential trends in market demand. Based on the analysis results, generate actionable strategy recommendations.

2. The e-commerce big data analysis method based on artificial intelligence according to claim 1, characterized in that, The specific steps for preprocessing e-commerce data from different sources and in different formats, and for building an integrated data warehouse, described in Step I, are as follows: S1.1: Randomly extract multiple sets of e-commerce data from various data sources, and list the field set and field example values ​​for each source. Then calculate the similarity between the field names and example values ​​of any two data sources. If the similarity is higher than the preset threshold, the field pair is listed as an automatic mapping candidate and a field mapping candidate pair is established. Otherwise, a mapping missing report is generated and marked as "requires manual review". S1.2: Perform distribution checks on the numerical fields in each e-commerce data and identify outliers. At the same time, generate an anomaly list to be manually reviewed. After the deduplication and anomaly handling of each e-commerce data are completed, generate and record the corresponding change log. S1.3: Fill the missing values ​​in the category fields of each e-commerce data by adding a new default category, then fill the missing values ​​in the numerical fields of each e-commerce data by using linear time series interpolation, then unify the time fields in each e-commerce data, and convert all time fields into standard numerical timestamps. S1.4: Unify the units of numerical fields in e-commerce data, convert fields with different units of measurement according to the mapping table, select the z-score standardization method to standardize the numerical fields, record the parameters used for conversion for each field, select the corresponding encoding method for the category field according to the corresponding cardinality, and save the mapping table as the corresponding metadata. S1.5: Based on historical experience or manual settings of primary key merging strategies for e-commerce data from various data sources, and in accordance with the preset primary key merging strategy, align and merge records from different sources that point to the same business entity. At the same time, when the same attribute conflicts in different sources, synthesize the final value according to the weight priority strategy, while retaining source traceability information and uncertainty measurement. S1.6: Write the merged e-commerce data into the data warehouse and add corresponding metadata for each piece of e-commerce data. The metadata includes the source list, merging strategy, cleaning log, last update timestamp and quality score.

3. The e-commerce big data analysis method based on artificial intelligence according to claim 2, characterized in that, The specific steps for constructing personalized user tags as described in step II are as follows: S2.1: Extract and preprocess user behavior data from e-commerce data from various data sources, sort all original behavior records of each user in chronological order, calculate the time gap between two adjacent records, and determine whether they are the same session or segment. If the time gap exceeds the preset idle threshold, it is considered a session boundary. Based on the identified session boundary, the session can be divided into multiple segments. Then, add the corresponding session identifier and segment number to each processed behavior record. S2.2: Calculate the basic behavioral features corresponding to session level, user level and time window level respectively, then calculate the granularity and time window boundary of each basic behavioral feature, while retaining the original distribution statistics within the time window, create an intermediate feature table, record each basic behavioral feature, and then calculate the recording time and sample size of each feature in the intermediate feature table. S2.3: Map user behavior data to corresponding event embedding sequences, count the frequency of state transitions within the same session or across sessions, and construct an empirical transition probability matrix to obtain the corresponding behavior path. Smooth and regularize the empirical transition probability matrix, and establish multiple sets of different types of input sequences. Then, select and train the corresponding sequence model according to the business objectives. S2.4: Based on the sequence model type after training, input the corresponding input sequence. Then, the sequence model outputs multiple sets of behavior vectors through forward propagation, and performs clustering or dimensionality reduction analysis on each set of behavior vectors to obtain the corresponding behavior prototype. Then, set a label set and design a corresponding mapping rule for each label. S2.5: A non-linear scoring function is used to map user behavior vectors to tag scores, and confidence and backtracking basis are calculated for each tag. If the tag score is greater than or equal to a preset threshold, a personalized tag for the user is directly generated based on the user behavior. Otherwise, the current user behavior is marked as pending manual review, and the tag timestamp and source are recorded.

4. The e-commerce big data analysis method based on artificial intelligence according to claim 3, characterized in that, The specific steps for extracting users' psychological needs, risk perceptions, and consumption intentions as described in Step III are as follows: S3.1: Perform language recognition and unified character encoding on various types of user text data respectively, then record the annotation language of non-target language and select translation or removal based on platform requirements, then remove HTML tags, URLs, repeated whitespace, and non-semantic noise symbols from each text data, and then replace special placeholders in user input with unified symbols. S3.2: Perform word segmentation and sub-word segmentation on the text data respectively, generate corresponding word sequence and record the original offset of the word sequence, then unify the capitalization of each word and sentence in the word sequence, remove stop words, then extract the stem of each word and sentence, and generate a normalized word sequence, original offset index, language identifier and preprocessing log for each text data. S3.3: Extract multiple sets of semantic features from each preprocessed text data through a semantic encoder, and perform batch normalization and PCA dimensionality reduction on each set of semantic features. Then, record each processed semantic feature in the feature index, and record the generator version, weight ID and normalization parameters. S3.4: A multimodal semantic discriminator is used for each semantic feature to generate confidence scores and thresholded judgment results for various labels of intent recognition, sentiment tendency and risk perception detection, while marking the key words or sentence positions of each label for recognition; S3.5: Extract entity categories from text data and record the character offset and context window of each entity. Calculate the context compatibility and string similarity between various data and the extracted entities in the platform's product knowledge base, brand library, common phrase table and external knowledge base. If the context compatibility or string similarity is higher than the preset threshold, mark the corresponding data as a candidate and generate a candidate list. S3.6: Calculate the confidence score of each entity-candidate pair, and select the entity-candidate pair with the highest confidence score as the optimal mapping. At the same time, mark entities that fail to match as "multiple candidates require manual review". Then output the final entity identifier and traceability information. Based on the text features and entity identifiers, evaluate the semantic matching degree and infer the implicit relationship pointed to by the text. Calculate the matching score of each text feature for the entity identifier, and based on the obtained matching score, generate a structured ternary expression of the corresponding user psychological needs, risk perception and consumption intention.

5. An artificial intelligence-based e-commerce big data analysis system, used to implement the artificial intelligence-based e-commerce big data analysis method according to any one of claims 1-4, characterized in that, It includes a data acquisition and processing module, a storage management module, a behavior analysis module, a cognitive recognition module, a fusion analysis module, a product analysis module, an inference and recognition module, a trend analysis module, a graph construction module, a risk warning module, a decision recommendation module, and an interactive monitoring module; The data acquisition and processing module is used to collect e-commerce data from multiple data sources in real time and preprocess the collected e-commerce data. The storage management module utilizes a distributed storage and data lake architecture to classify, store, and manage the pre-processed e-commerce data in a unified manner. The behavior analysis module is used to extract user behavior features from various behavior information and identify user behavior patterns; The cognitive recognition module is used to analyze user comments, search terms, and consultation content to construct the user's cognitive characteristics; The fusion analysis module decouples and analyzes user motivation based on user behavior characteristics and cognitive characteristics; The product analysis module is used to analyze product attribute tags, price curves, evaluation characteristics, and competitive environment; The inference and identification module is used to infer macroeconomic factors and individual user behavior respectively, and to identify each influencing factor; The trend analysis module is used to identify changes in demand, evolution of user sentiment, and trends in various e-commerce information throughout the product lifecycle. The graph construction module establishes a cognitive evolution path graph based on changes in user cognition; The risk warning module detects potential risks based on trend analysis and inference results, while identifying market opportunities and providing early warnings. The decision recommendation module generates executable optimization strategies based on various analysis results; The interactive monitoring module is used to visualize the output of each module.

6. The e-commerce big data analysis system based on artificial intelligence according to claim 5, characterized in that, The specific steps of the fusion analysis module in decoupling and analyzing user motivation are as follows: S4.1: Based on the user's behavioral and cognitive characteristics, establish corresponding behavioral and cognitive graphs, and map user behavior and cognition to one or more nodes on the behavioral and cognitive graphs. Generate initial edge candidates for the behavioral and cognitive graphs by statistically counting co-occurrence and context window co-occurrence rate. S4.2: Filter out candidate edges whose confidence does not meet the preset threshold, and initialize weight and time label for each remaining edge to establish two adjacency lists and a node attribute table. The adjacency list includes a behavioral edge list and a semantic edge list. S4.3: Set the feature clusters of each node in the behavior graph and cognitive graph, then concatenate the multimodal features and map them to a unified dimension through a small feedforward network to generate the initial node features corresponding to each node. Then, according to the preset requirements, determine the connection edge features between each node and generate the corresponding edge features through normalization and discrete encoding. S4.4: Input the behavior graph and cognitive graph into the heterogeneous graph neural network. At the same time, the heterogeneous graph neural network extracts the adjacency structure, initial node features and edge features from each user's behavior graph and cognitive graph in parallel. Through the forward propagation algorithm, it performs the steps of collecting messages from neighboring nodes, updating node representations based on edge features and weighted aggregation, and outputs the prediction probability of the user's next behavior and the prediction of topic tags. Based on the prediction results, it generates the corresponding behavior driving components and cognitive driving components for the user and records the contribution of each driving component to the current user.

7. The e-commerce big data analysis system based on artificial intelligence according to claim 5, characterized in that, The inference and identification module infers macroeconomic factors and individual user behavior, and the specific steps for identifying each influencing factor are as follows: S5.1: The indicators to be monitored and the required intermediate variables from the behavioral and cognitive paths are used as the node set of the graph. The co-occurrence and sequential occurrence counts of nodes in each time window are counted. Then, the co-occurrence probability and marginal probability of each pair of nodes in each time window are calculated to obtain the corresponding candidate causal strength. If there is a candidate causal strength lower than the preset confidence threshold, the edge between the corresponding nodes is removed, and the candidate causal adjacency matrix at the corresponding time is generated. At the same time, the causal graph is constructed. S5.2: At each time point, the target is set to be jointly acted by candidate parent nodes in a linear or generalized linear manner, and the structural coefficient matrix of the corresponding time is calculated in the sliding window through a regression form containing smoothing constraints. The significance of each calculated structural coefficient is tested, and the candidate edges that pass the test are used as real-time causal edges. S5.3: Based on the newly generated structural coefficient matrix, update the weights of each connecting edge and the local propagation rules in the causal graph, then update the hidden representations of each node and edge in the causal graph according to the current node and edge features, and output the prediction and influence propagation mode in real time, while generating short-term causal propagation paths. S5.4: Collect all user driving factors, randomly select any driving factor for random perturbation, and predict the changes of the corresponding target at a preset future time point through a graph propagation model. At the same time, use Monte Carlo simulation to calculate the distribution and uncertainty of the prediction results, and output the expected causal effect and confidence interval. S5.5: Sort the expected causal effects of each candidate driving factor from high to low according to the absolute value, output a real-time list of key factors, and record the confidence level and main propagation path of each driving factor. Then set the business threshold and combine the ranking results with the threshold rules to form an automatic alarm rule.

Images