Big data intelligent analysis method and system based on AI drive

Abstract

The invention relates to the technical field of big data processing, in particular to an intelligent big data analysis method and system based on AI driving. The method comprises the following steps: acquiring a user behavior activity description text big data set, performing behavior text semantic feature analysis, and performing embedded and filtering feature selection processing at the same time to obtain a user behavior semantic embedded feature subset and a user behavior semantic filtering feature subset; performing repeated representative feature screening processing and behavior activity feature classification according to the user behavior semantic embedded feature subset and the user behavior semantic filtering feature subset to obtain a behavior semantic representative feature subset corresponding to each user behavior activity, and carrying out local interpretability AI intelligent analysis and logic reason visualization display processing on the user behavior activity decision to obtain an interpretability logic reason behind each user behavior activity decision. According to the invention, the efficiency and accuracy of big data analysis can be improved.

Description

Technical Field

[0001] The present invention relates to the field of big data processing technology, and in particular to an AI-driven big data intelligent parsing and analysis method and system. Background Art

[0002] In the digital age, the explosive growth of big data has brought unprecedented challenges and opportunities to all walks of life. The generation and storage of massive user behavior activity data provides a rich foundation for the optimization of big data services. Through automated data integration technology, data from different sources (such as databases, sensors, text files, social media, etc.) are effectively aggregated and cleaned to ensure data quality and consistency. This process includes steps such as removing duplicate data, filling missing values, and formatting, laying a solid foundation for subsequent analysis. At the same time, by using advanced machine learning and deep learning algorithms, large-scale user behavior activity data can be efficiently modeled and analyzed. These algorithms can automatically identify complex patterns in big data and capture nonlinear relationships, thereby improving the accuracy of big data intelligent analysis. However, traditional big data analysis and processing methods often rely on rules and statistical models, which are difficult to adapt to the diversity and complexity of big data, resulting in low efficiency of big data intelligent analysis, and thus unable to effectively support user behavior activity decision-making. Summary of the invention

[0003] Based on this, it is necessary for the present invention to provide an AI-driven big data intelligent parsing and analysis method and system to solve at least one of the above technical problems.

[0004] To achieve the above purpose, a big data intelligent parsing and analysis method based on AI drive includes the following steps: Step S1: obtaining a large dataset of user behavior activity description texts, and performing behavior text semantic feature analysis on the large dataset of user behavior activity description texts to obtain a user behavior activity description text semantic feature set; Step S2: performing embedded and filtered feature selection processing on the semantic feature set of the user behavior activity description text to obtain a user behavior semantic embedded feature subset and a user behavior semantic filtered feature subset; performing repeated representative feature screening processing on the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset to obtain a user behavior semantic comprehensive representative feature set; Step S3: classify the behavior activity features of the user behavior semantics comprehensive representative feature set to obtain the behavior semantics representative feature subset corresponding to each user behavior activity; perform local interpretable AI intelligent parsing analysis on the behavior semantics representative feature subset corresponding to each user behavior activity to obtain the user behavior decision interpretability factor parsing result corresponding to each user behavior activity; Step S4: Perform a logical reason visualization processing on the user behavior decision explainability factor analysis results corresponding to each user behavior activity to obtain the explainable logical reasons behind each user behavior activity decision.

[0005] Further, step S1 includes the following steps: Step S11: obtaining a large data set of user behavior activity description texts, wherein the large data set of user behavior activity description texts includes user behavior activity text record data from user behavior data sources related to social media, mobile applications, online platforms, and e-commerce websites; Step S12: performing sensitivity elimination processing on the user behavior activity description text big data set to obtain the user behavior activity text sensitive word elimination big data set; performing stop word removal and text normalization processing on the user behavior activity text sensitive word elimination big data set to obtain the standardized user behavior activity text big data set; Step S13: extracting semantic atomic units from the standardized user behavior activity text big data set to obtain a set of semantic component atomic units of the user behavior activity text; Step S14: performing text semantic relationship mining and analysis on each text basic semantic component atomic unit in the user behavior activity text semantic component atomic unit set to obtain the user behavior activity text semantic relationship between each text basic semantic component atomic unit; constructing a semantic relationship graph for each corresponding text basic semantic component atomic unit based on the user behavior activity text semantic relationship between each text basic semantic component atomic unit to generate a user behavior activity text semantic relationship graph; Step S15: Based on the user behavior activity text semantic relationship graph, semantic feature recognition analysis is performed on the corresponding user behavior activity description text in the standardized user behavior activity text big data set to obtain a semantic feature set of the user behavior activity description text.

[0006] Furthermore, the step S14 of constructing a semantic relationship graph for each corresponding text basic semantic component atomic unit based on the user behavior activity text semantic relationship between each text basic semantic component atomic unit includes the following steps: Perform text frequency and reverse document frequency statistics calculation on each text basic semantic component atomic unit in the user behavior activity text semantic component atomic unit set, so as to obtain the text frequency and reverse document frequency corresponding to each text basic semantic component atomic unit; Based on the text frequency and reverse document frequency corresponding to each text basic semantic component atomic unit, the semantic relationship weight distribution calculation of the user behavior activity text semantic relationship between each text basic semantic component atomic unit is performed to obtain the text semantic relationship distribution weight between each text basic semantic component atomic unit; According to the text semantic relationship allocation weights between the basic semantic component atomic units of each text, the user behavior activity text semantic relationship between the basic semantic component atomic units of each text is determined by semantic relationship weight mapping, so as to obtain the user activity text semantic relationship weight mapping rules between the basic semantic component atomic units of each text; Based on the user activity text semantic relationship weight mapping rules between each text basic semantic component atomic unit, the user behavior activity text semantic relationship between each text basic semantic component atomic unit and each corresponding text basic semantic component atomic unit are constructed to generate a user behavior activity text semantic relationship map.

[0007] Further, step S2 includes the following steps: Step S21: performing embedded feature selection processing on the semantic feature set of the user behavior activity description text to obtain a user behavior semantic embedded feature subset; Step S22: performing filtering feature selection processing on the semantic feature set of the user behavior activity description text to obtain a user behavior semantic filtering feature subset; Step S23: performing repeated feature screening processing according to the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset to obtain a user behavior semantic repeated feature set; Step S24: using the behavior semantic representativeness calculation formula to perform representativeness score calculation on each user behavior semantic sub-feature in the user behavior semantic repeated feature set, so as to obtain a feature representativeness score value corresponding to each user behavior semantic sub-feature; The specific calculation formula for the representativeness of behavioral semantics is: ; In the formula, For the User behavior semantic sub-features The corresponding feature representative score value, For the User behavior semantic sub-features, For the User behavior semantic sub-features, and are all item index parameters of user behavior semantic sub-features, is the total number of semantic sub-features of user behavior, is the normalization constant, is the lower limit of the integration time range, is the upper limit of the integration time range, is the time variable parameter, For the The weight parameter corresponding to the semantic sub-feature of user behavior, is an exponential function, For the The feature center position corresponding to each user behavior semantic sub-feature in the user behavior semantic repeated feature set, For the The degree of feature dispersion corresponding to each user behavior semantic sub-feature in the user behavior semantic repeated feature set, For the The semantic sub-features of user behavior and User behavior semantic sub-features at time point The semantic feature similarity measure at is the correction factor of the characteristic representativeness score; Step S25: Compare and judge the feature representativeness score value corresponding to each user behavior semantic sub-feature according to the preset representativeness score threshold; when the feature representativeness score value is greater than or equal to the preset representativeness score threshold, the corresponding user behavior semantic sub-feature is marked as a representative feature; when the feature representativeness score value is less than the preset representativeness score threshold, the corresponding user behavior semantic sub-feature is marked as a non-representative feature; the user behavior semantic sub-features marked as representative features in the user behavior semantic repeated feature set are screened out and combined to obtain a comprehensive representative feature set of user behavior semantics.

[0008] Further, step S21 includes the following steps: Perform feature importance evaluation and analysis on each user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain the feature importance degree corresponding to each user behavior semantic sub-feature; Perform feature sparsity quantification calculation on each user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain the feature sparsity degree corresponding to each user behavior semantic sub-feature; According to the feature importance and feature sparsity of each user behavior semantic sub-feature, L1 regularization statistical analysis is performed on each user behavior semantic sub-feature corresponding to the user behavior activity description text semantic feature set to obtain the L1 regularization strength parameter of the user behavior semantic feature; Based on the L1 regularization strength parameter of the user behavior semantic feature, L1 regularized embedded feature selection is performed on each corresponding user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain the user behavior semantic embedded feature subset.

[0009] Further, step S22 includes the following steps: Performing statistical calculations on the number of occurrences of activity descriptions for each user behavior semantic sub-feature in the user behavior activity description text semantic feature set, and obtaining the number of occurrences of user behavior activity descriptions corresponding to each user behavior semantic sub-feature; Perform feature frequency statistical analysis on each corresponding user behavior semantic sub-feature according to the number of occurrences of the user behavior activity description corresponding to each user behavior semantic sub-feature, and obtain the user behavior semantic feature frequency corresponding to each user behavior semantic sub-feature; Performing feature correlation evaluation and analysis on each user behavior semantic sub-feature in the user behavior activity description text semantic feature set to obtain the correlation coefficient between each user behavior semantic sub-feature and the user behavior activity description text; setting the user behavior activity significance level according to the correlation coefficient between each user behavior semantic sub-feature and the user behavior activity description text to obtain the user behavior activity semantic feature significance level; Based on the significance level of the semantic feature of the user behavior activity and the user behavior semantic feature frequency corresponding to each user behavior semantic sub-feature, a sub-feature chi-square test is performed on each corresponding user behavior semantic sub-feature to obtain the semantic feature chi-square value corresponding to each user behavior semantic sub-feature; Based on the semantic feature chi-square value corresponding to each user behavior semantic sub-feature, a filtering feature selection process is performed on each corresponding user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain a user behavior semantic filtering feature subset.

[0010] Further, step S3 includes the following steps: Step S31: extracting context information from each user behavior semantics representative sub-feature in the user behavior semantics comprehensive representative feature set to obtain user behavior semantics context information corresponding to each user behavior semantics representative sub-feature; Step S32: extracting sub-feature activity attributes from each corresponding user behavior semantic representative sub-feature based on the user behavior semantic context information corresponding to each user behavior semantic representative sub-feature, and obtaining the user behavior activity attributes corresponding to each user behavior semantic representative sub-feature; Step S33: classifying each corresponding user behavior semantics representative sub-feature in the user behavior semantics comprehensive representative feature set based on the user behavior activity attribute corresponding to each user behavior semantics representative sub-feature, so as to obtain a behavior semantics representative feature subset corresponding to each user behavior activity; Step S34: performing local interpretability evaluation and analysis on the behavioral semantic representative feature subset corresponding to each user behavior activity to obtain a local interpretability factor of the user behavior decision corresponding to each user behavior activity; Step S35: Perform AI-driven intelligent parsing analysis on the local explainability factors of user behavior decisions corresponding to each user behavior activity to obtain the parsing results of the explainability factors of user behavior decisions corresponding to each user behavior activity.

[0011] Further, step S34 includes the following steps: Step S341: performing user behavior decision semantic association analysis on each behavior semantic representative sub-feature in the behavior semantic representative feature subset corresponding to each user behavior activity, to obtain the semantic association relationship between the behavior semantic representative sub-feature corresponding to each user behavior activity and the user behavior decision; Step S342: Based on the semantic association relationship between the behavioral semantic representative sub-features corresponding to each user behavior activity and the user behavior decision, a behavior decision local influencing factor analysis is performed on the behavior semantic representative feature subset corresponding to each user behavior activity to obtain a user behavior decision local influencing factor candidate set corresponding to each user behavior activity; Step S343: performing user behavior performance evaluation and analysis on each behavior semantics representative sub-feature in the behavior semantics representative feature subset corresponding to each user behavior activity, and obtaining a user behavior activity performance score corresponding to each behavior semantics representative sub-feature of each user behavior activity; Step S344: Based on the user behavior activity performance score corresponding to each behavior semantic representative sub-feature of each user behavior activity, a local interpretability framework design is performed on the behavior semantic representative feature subset corresponding to each user behavior activity to generate a local interpretability evaluation and analysis model corresponding to each user behavior activity; Step S345: Perform local interpretability evaluation and analysis on the candidate set of local influencing factors of user behavior decisions corresponding to each user behavior activity according to the local interpretability evaluation and analysis model corresponding to each user behavior activity, and obtain the local interpretability factor of user behavior decision corresponding to each user behavior activity.

[0012] Further, step S4 includes the following steps: Step S41: performing factor correlation statistical analysis on the corresponding user behavior decision interpretability factors in the user behavior decision interpretability factor analysis results corresponding to each user behavior activity, and obtaining the correlation between the decision interpretability factors corresponding to each user behavior activity; Step S42: constructing a decision logic relationship network for the corresponding user behavior decision interpretability factors based on the correlation between the decision interpretability factors corresponding to each user behavior activity, so as to generate a user behavior decision logic relationship network structure corresponding to each user behavior activity; Step S43: Based on the user behavior decision logical relationship network structure corresponding to each user behavior activity, the user behavior decision interpretability factor analysis results corresponding to each user behavior activity are visualized and displayed with logical reasons to obtain the interpretable logical reasons behind each user behavior activity decision.

[0013] Furthermore, the present invention also provides an AI-driven big data intelligent parsing and analysis system for executing the AI-driven big data intelligent parsing and analysis method as described above. The AI-driven big data intelligent parsing and analysis system includes: A user behavior semantic feature analysis module is used to obtain a large data set of user behavior activity description texts, and perform behavior text semantic feature analysis on the large data set of user behavior activity description texts, thereby obtaining a user behavior activity description text semantic feature set; The user behavior semantic repeated representative screening module is used to perform embedded and filtered feature selection processing on the semantic feature set of the user behavior activity description text to obtain the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset; perform repeated representative feature screening processing based on the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset, so as to obtain the user behavior semantic comprehensive representative feature set; The behavior activity AI-driven intelligent analysis module is used to classify the behavior activity features of the user behavior semantics comprehensive representative feature set to obtain the behavior semantics representative feature subset corresponding to each user behavior activity; perform local interpretable AI intelligent analysis on the behavior semantics representative feature subset corresponding to each user behavior activity to obtain the user behavior decision interpretability factor analysis result corresponding to each user behavior activity; The behavior decision analysis reason visualization module is used to visualize the logical reasons of the user behavior decision explainability factor analysis results corresponding to each user behavior activity, so as to obtain the explainable logical reasons behind each user behavior activity decision.

[0014] Beneficial effects of the present invention: 1. Compared with the prior art, the AI-driven big data intelligent parsing and analysis method proposed in the present invention has the beneficial effect of providing rich materials for subsequent analysis by collecting large data sets of text records related to user behavior activities from multiple channels such as social media, mobile applications, online platforms and e-commerce websites. User interaction records on social media, user operation logs in mobile applications, comments and feedback on online platforms, and purchase and browsing records on e-commerce websites are all true reflections of user behavior activities. By aggregating these data, users' needs, preferences and behavior patterns can be fully understood, thereby providing support for the subsequent user behavior activity decision parsing and analysis process. The huge amount of data also lays the foundation for the application of algorithms such as data mining, machine learning and deep learning, making the analysis results more accurate and reliable. At the same time, by performing behavior text semantic feature analysis on a large data set of user behavior activity description texts to analyze and identify the semantic features in the user behavior activity description texts, a set of key features reflecting user needs and preferences can be effectively extracted. These features can not only help better understand the actual needs of users, but also provide a scientific basis for subsequent processing processes. Secondly, by performing embedded feature selection on the semantic feature set of the user behavior activity description text, the aim is to automatically identify the features that have the greatest impact on subsequent analysis through L1 regularization technology. Embedded feature selection not only relies on the independence or importance score of the feature, but also combines the model training process, so that feature selection and model construction are carried out simultaneously, thereby improving the efficiency and effectiveness of feature selection, and can also significantly reduce redundant and irrelevant features, reduce the complexity of text semantic features, and thus provide a reliable basis for subsequent analysis and decision-making. The semantic feature set of the user behavior activity description text is also processed by using a filtering feature selection method. Unlike embedded feature selection, filtering feature selection is usually performed before model training. By performing independent statistical analysis on each feature in the feature set, using statistics (such as chi-square test, information gain or mutual information, etc.) to evaluate the relationship between the feature and the target variable, the features most relevant to the user behavior are screened out. The advantage of this process is that it has high computational efficiency and does not rely on the training of any machine learning model, so that it can be quickly executed even on large-scale data sets, which can effectively remove noise features, thereby providing a clearer and more concise feature set for subsequent processing.By screening repeated features based on the user behavior semantic embedded feature subset and the user behavior semantic filtering feature subset, the aim is to further screen out the repeated features in the feature set. By cross-comparing the two feature subsets, we can identify which features appear in different selection methods and screen them out for backup. By setting a reasonable threshold, we can effectively screen out the key features that can accurately reflect user behavior, while excluding the features that do not contribute much to the model. Such a selection mechanism can ensure the simplicity and effectiveness of the final feature set, while reducing the complexity of model training. This not only improves the accuracy of subsequent data analysis, but also provides a more scientific and reliable basis for the subsequent big data intelligent analysis and processing process. Then, by classifying the behavioral activity features of the user behavior semantic comprehensive representative feature set, the value of this step is that it can structure the massive user behavior information so that different types of behaviors can be effectively classified. The classification process not only helps to identify the commonality and individuality of user behavior, but also reveals the potential correlation between behaviors. This analysis can provide a basis for user stratification, thereby providing basic data guarantee for the subsequent analysis and processing process. By conducting local interpretability evaluation and analysis on the behavioral semantic representative feature subsets corresponding to each user behavior activity and performing AI-driven intelligent parsing analysis, we can further deepen our understanding of user behavior. This process converts the interpretability factors of user behavior decisions into actionable parsing results through machine learning and data analysis technology. The key to this analysis is that it can provide clear user behavior insights for subsequent processing. The AI-driven parsing process can respond quickly to market changes and adjust strategies in a timely manner to cope with changes in user needs, formulate more targeted and timely strategies, and ensure that competitive advantages can be maintained at every stage of user decision-making, thereby improving the efficiency of the big data intelligent parsing process. Finally, the logical reasons are visualized by analyzing the interpretability factor analysis results of user behavior decisions corresponding to each user behavior activity. The key to this step is that it not only provides an intuitive understanding of user behavior decisions, but also promotes effective communication with decision makers. The visualization of logical reasons transforms complex data and relationships into easy-to-understand graphics, which greatly reduces the difficulty of information communication. Through graphical methods, decision makers can quickly grasp key factors and their relationships, which facilitates decision-making and action. Compared with traditional text reports, this visualization method can better attract the audience's attention and improve the efficiency of information dissemination. It can also help decision makers consider more dimensional factors when formulating strategies. For example, in industries such as e-commerce and finance, decisions often involve trade-offs in multiple aspects. Being able to clearly see the relationship between factors helps to comprehensively evaluate the corresponding impact results. This systematic way of thinking promotes a more rational decision-making process and reduces the probability of decision-making errors, thereby effectively supporting user behavior activity decisions.

[0015] 2. The AI-driven big data intelligent parsing and analysis system proposed in the present invention is composed of a user behavior semantic feature analysis module, a user behavior semantic repeated representative screening module, a behavior activity AI-driven intelligent parsing module, and a behavior decision parsing reason visualization module. It can realize any AI-driven big data intelligent parsing and analysis method described in the present invention, and is used to combine the operations between computer programs running on each module to realize the AI-driven big data intelligent parsing and analysis method. The internal structures of the system cooperate with each other, which can greatly reduce duplication of work and manpower investment, and can quickly and effectively provide a more accurate and efficient AI-driven big data intelligent parsing and analysis process, thereby simplifying the operation flow of the AI-driven big data intelligent parsing and analysis system. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings: Figure 1 This is a schematic diagram of the steps of the AI-driven big data intelligent parsing and analysis method of the present invention; Figure 2 for Figure 1 Detailed step flow diagram of step S1; Figure 3 for Figure 1 Detailed step flow chart of step S2 in FIG. DETAILED DESCRIPTION

[0017] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0018] To achieve this, please refer to Figures 1 to 3 The present invention provides an AI-driven big data intelligent analysis method. In the embodiments of the present invention, please refer to Figure 1 FIG. 1 is a schematic diagram of the steps of the AI-driven big data intelligent analysis method of the present invention, and the AI-driven big data intelligent analysis method includes the following steps: Step S1: obtaining a large dataset of user behavior activity description texts, and performing behavior text semantic feature analysis on the large dataset of user behavior activity description texts to obtain a user behavior activity description text semantic feature set; In an embodiment of the present invention, descriptive text information data related to user behavior activities is obtained in real time from social media platforms (such as Twitter, Facebook, Weibo), mobile applications (such as logs within apps), online platforms (such as forums, blogs), and e-commerce websites (such as user comments and feedback on e-commerce platforms). The user behavior activity text data on various platforms are batch captured through API interfaces or data crawler technology. These data include text descriptions of users' posting records, comment content, purchase behaviors, browsing records, likes, shares, and other behaviors on the platforms, thereby obtaining a large data set of user behavior activity description texts. By eliminating sensitive words, removing stop words and normalizing the text of the previously collected large data set of user behavior activity descriptions, and using a unified word segmentation tool, such as Jieba word segmenter or SpaCy, to segment words, the special characters and punctuation marks in the text are uniformly formatted to ensure the consistency and standardization of the text in subsequent processing. The text is split into vocabulary units by using the word segmentation tool in natural language processing technology, and the word level segmentation is performed using Jieba word segmentation or SpaCy to ensure that the granularity of the semantic unit is sufficiently refined. In combination with dependency syntactic analysis technology, semantic components such as nouns and verbs in the text are extracted to construct a set of semantic atomic units. At the same time, the semantic relationship between each basic semantic component atomic unit of the text in the previously extracted semantic component atomic unit set is mined and analyzed to discover the potential semantic associations between the basic semantic component atomic units of the text, and each basic semantic component atomic unit is regarded as a node in the graph, while the semantic relationship is regarded as the edge between the nodes. In addition, when constructing the graph, the weight of the edge is derived from the similarity of the semantic relationship mining results, that is, the edge weight between the semantic component pairs with higher relationship strength is greater, thereby converting the text data and semantic relationship into a graph structure to ensure that the semantic relationship structure of the user behavior activity is clear and operational. Then, by combining the previously constructed semantic relationship graph, the corresponding user behavior activity description text is identified and analyzed for semantic features. The text is grouped through the relationship graph stored in the graph database and the community detection algorithm in graph theory (such as Louvain algorithm or spectral clustering) to identify user groups with similar behaviors in a specific semantic field. In the feature extraction process, machine learning models (such as support vector machines SVM or random forests) are used to classify the nodes in the graph and extract the main semantic features related to user behavior activities. Through feature engineering technology, the feature set is further screened and optimized to obtain a set of semantic feature sets that are helpful for further user behavior analysis and prediction, and finally the user behavior activity description text semantic feature set is obtained.

[0019] Step S2: performing embedded and filtered feature selection processing on the semantic feature set of the user behavior activity description text to obtain a user behavior semantic embedded feature subset and a user behavior semantic filtered feature subset; performing repeated representative feature screening processing on the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset to obtain a user behavior semantic comprehensive representative feature set; In an embodiment of the present invention, by using a pre-trained deep learning model, such as BERT or Word2Vec, the corresponding user behavior semantic sub-features in the semantic feature set of the user behavior activity description text are vectorized, and the user behavior semantic sub-features are converted into dense vector representations in a high-dimensional feature space. By analyzing these vectors, the relationship strength between each feature and the target variable can be calculated, and the L1 regularization method (Lasso regression) is used to perform feature selection, retaining features with high correlation with user behavior performance, and eliminating redundant and irrelevant features, thereby obtaining a user behavior semantic embedded feature subset. In addition, by performing filtered feature selection on the corresponding user behavior semantic sub-features in the semantic feature set of the user behavior activity description text, the independence between each user behavior semantic sub-feature and the user behavior result is evaluated by using the statistical method of the chi-square test, and by calculating the chi-square value of each user behavior semantic sub-feature, the features with a chi-square value higher than a preset threshold are screened out, thereby obtaining a user behavior semantic filtered feature subset. At the same time, repeated features are screened for the previously discriminated and screened user behavior semantic embedded feature subset and the filtered feature subset. At this time, the two feature subsets are intersected to further screen out those features that have obvious overlap in user behavior semantics and remove their duplicates. The representative score calculation is also performed on each user behavior semantic sub-feature in the previously screened semantic duplicate feature set to ensure that the representative score of each sub-feature is accurately calculated. Then, the feature representative score value obtained by the previous quantitative calculation is compared and judged by using a preset representative score threshold to set a reasonable score threshold. The value is based on the analysis results of the feature performance in the previous step to ensure that it can effectively distinguish between representative features and non-representative features. Subsequently, the feature representative score value corresponding to each user behavior semantic sub-feature is traversed, and the representative features (if the feature representative score value is greater than or equal to the preset representative score threshold) and non-representative features (if the feature representative score value is less than the preset representative score threshold) are marked through conditional judgment statements, and the user behavior semantic sub-features marked as representative features are screened out and integrated into a data set to finally obtain a comprehensive representative feature set of user behavior semantics.

[0020] Step S3: classify the behavior activity features of the user behavior semantics comprehensive representative feature set to obtain the behavior semantics representative feature subset corresponding to each user behavior activity; perform local interpretable AI intelligent parsing analysis on the behavior semantics representative feature subset corresponding to each user behavior activity to obtain the user behavior decision interpretability factor parsing result corresponding to each user behavior activity; In an embodiment of the present invention, context information is extracted from each user behavior semantic representative sub-feature in the user behavior semantic comprehensive representative feature set obtained by previous analysis. These sub-features include user clicks, search terms, purchase records, etc., and natural language processing (NLP) technology is used to extract context information related to these sub-features by adopting methods such as word segmentation, part-of-speech tagging and named entity recognition. For example, for "the user clicked on the link of product A", keywords such as "user", "click", "product A" and "link" can be extracted to generate context information about the behavior, including time, location, device type and other information. Sub-feature activity attributes are extracted from each corresponding user behavior semantic representative sub-feature by combining the user behavior semantic context information extracted previously, so as to obtain "user clicked on the link of product A". Taking the behavior of "clicking on the link of product A" as an example, the activity attributes may include "product type", "click frequency", "user group", etc., so as to summarize the activity attributes of each sub-feature. At the same time, by combining the user behavior activity attributes corresponding to each representative sub-feature obtained by the previous mining analysis, the behavior activity features of each user behavior semantic representative sub-feature corresponding to the user behavior semantic comprehensive representative feature set are classified. In this step, the user behavior activities can be classified with the help of machine learning models (such as support vector machines SVM or random forests). The activity attributes of each sub-feature are used as input features. After training, the model can identify the categories of different user behavior activities, such as "browsing", "purchasing", "commenting", etc., so as to classify the behavior semantic representative feature subsets corresponding to each user behavior activity.Then, we evaluate and analyze the local interpretability of each user behavior activity and its corresponding subset of representative behavioral semantic features. In this process, we use local interpretable model-agnostic explanation methods (such as LIME or SHAP) to identify the key factors that affect user decisions. By generating a local model, we analyze the relationship between user behavior activity characteristics and decision results, and provide interpretable factors for each decision. For example, if a user chooses to buy a product, LIME can help identify the role of features such as "discounts" and "product reviews" in the decision. This process needs to be combined with data visualization tools (such as Matplotlib or Tableau) to display the interpretable analysis results, so that the local interpretable factors of the decision are clear at a glance, that is, the degree of influence of the corresponding local influencing factors in the user's decision-making process, and through AI-driven intelligent analysis is performed on the local interpretable factors of user behavior decisions obtained after previous evaluation and analysis. This step will combine deep learning technology (such as deep neural networks) for factor analysis, so as to use neural network models to conduct deeper feature learning of local interpretable factors, and use the model to perform nonlinear mapping of the relationship between factors to extract more complex decision logic. After the model is trained, the corresponding decision interpretable factor analysis results can be generated. For example, the model will point out that a user is more influenced by "social media influence" and "similar product reviews" when making a purchase. This process relies on deep learning frameworks (such as TensorFlow or PyTorch) and analysis tools to ensure that the final analysis results are highly credible and practical, and finally the user behavior decision interpretable factor analysis results corresponding to each user behavior activity are obtained.

[0021] Step S4: Perform a logical reason visualization processing on the user behavior decision explainability factor analysis results corresponding to each user behavior activity to obtain the explainable logical reasons behind each user behavior activity decision.

[0022] In an embodiment of the present invention, the Pearson correlation coefficient or the Spearman rank correlation coefficient is used to perform a statistical analysis of the correlation of the corresponding user behavior decision interpretability factors in the analysis results of the user behavior decision interpretability factors corresponding to each user behavior activity, so as to input the collected user behavior decision interpretability factors into statistical software or data analysis tools, such as the pandas library in Python or the psych package in the R language, and the correlation calculation function is used for calculation to obtain a correlation matrix between the factors, which clearly shows the correlation between the decision interpretability factors corresponding to each user behavior activity, and by combining the correlation between the decision interpretability factors corresponding to each user behavior activity obtained by the previous quantitative calculation, a logical relationship network is constructed for the corresponding user behavior decision interpretability factors, so as to utilize the concepts of nodes and edges in graph theory, regard the user behavior decision interpretability factors as nodes, and the correlation between the factors as the weight of the edge, and with the help of network analysis tools, such as Gephi or NetworkX, establish a directed graph containing all relevant factors, and set the weight of the edge to reflect the strength of its correlation, and then apply a network analysis algorithm (such as the Dijkstra algorithm or PageRank Algorithm), identify important factors and their relationships, generate the topological structure of the decision logic relationship network, and then, by combining the decision logic relationship network structure constructed by previous connections, visualize the logical reasons for the corresponding user behavior decision interpretability factor analysis results, so as to more intuitively understand the decision logic reasons behind user behavior activities. By selecting key factors based on the constructed decision logic relationship network and presenting their data in a graphical way, you can use chart tools (such as D3.js or Plotly) to create interactive visualizations to display the factors and their relationships and influences. By setting interactive functions, users can click on a factor to view its specific impact and logical path on the decision, and then analyze the importance of the factor in the decision-making process and its logical explanation reasons, so as to generate a corresponding decision logic display report, which will include a detailed analysis of the relationship between factors, an impact assessment and a comprehensive explanation of user behavior decisions, and finally obtain the interpretable logical reasons behind each user behavior activity decision.

[0023] Further, as an embodiment of the present invention, refer to Figure 2 As shown, Figure 1 Detailed step flow diagram of step S1 in FIG. 1 , in this embodiment, step S1 includes the following steps: Step S11: obtaining a large data set of user behavior activity description texts, wherein the large data set of user behavior activity description texts includes user behavior activity text record data from user behavior data sources related to social media, mobile applications, online platforms, and e-commerce websites; In the embodiments of the present invention, descriptive text information data related to user behavior activities is collected in real time from social media platforms (such as Twitter, Facebook, Weibo), mobile applications (such as in-app logs), online platforms (such as forums, blogs), and e-commerce websites (such as user comments and feedback on e-commerce platforms). The user behavior activity text data on various platforms is batch-crawled through API interfaces or data crawler technologies to ensure coverage of a wide range of user activity scenarios. These data include text descriptions of user posting records, comment content, purchase behaviors, browsing records, likes, shares, etc. To ensure the extensiveness and comprehensiveness of the data, the Requests library in the Python programming language is used for API calls, or the Scrapy framework is used for web data crawling, and the data is organized into a large dataset, and finally a large dataset of user behavior activity description texts is obtained.

[0024] Step S12: Perform sensitive information elimination processing on the large dataset of user behavior activity description texts to obtain a large dataset of user behavior activity texts with sensitive words eliminated; perform stop word removal and text normalization processing on the large dataset of user behavior activity texts with sensitive words eliminated to obtain a standardized large dataset of user behavior activity texts; In the embodiments of the present invention, by performing elimination processing on sensitive words in the previously collected large dataset of user behavior activity description texts, the sensitive word list is updated regularly through a standard thesaurus or set manually. Regular expressions are used to match sensitive words and replace them with harmless characters. This process is implemented using the re module in Python to ensure that sensitive information in the text data is removed, thereby obtaining a large dataset of user behavior activity texts with sensitive words eliminated. Then, stop word removal is performed on the large dataset of user behavior activity texts with sensitive words eliminated. The stop word list includes common words without substantial meaning (such as "of", "is", "in", etc.). These words are not helpful for semantic analysis. Therefore, the stop word list in the NLTK library is used for removal, and text normalization processing is performed on the large dataset of text after stop word removal. By using a unified word segmentation tool, such as the Jieba tokenizer or SpaCy, word segmentation is performed, and special characters and punctuation marks in the text are uniformly formatted to ensure the consistency and standardization of the text in subsequent processing. Finally, a standardized large dataset of user behavior activity texts is obtained.

[0025] Step S13: Perform semantic atomic unit extraction processing on the standardized large dataset of user behavior activity texts to obtain a set of semantic component atomic units of user behavior activity texts; In an embodiment of the present invention, semantic atomic units are extracted from a previously standardized large data set of standardized user behavior activity text, so as to split the text into vocabulary units by using a word segmentation tool in natural language processing technology, and use Jieba word segmentation or SpaCy to perform vocabulary-level segmentation to ensure that the granularity of the semantic units is sufficiently refined. Next, in combination with dependency syntactic analysis technology, semantic components such as nouns and verbs in the text are extracted to construct a set of semantic atomic units. This process is achieved by using the dependency parsing function in StanfordNLP or Spacy, ensuring that each semantic component unit is accurately extracted from the text, forming a set of atomic units with vocabulary as the core, and finally obtaining a set of semantic component atomic units of the user behavior activity text.

[0026] Step S14: performing text semantic relationship mining and analysis on each text basic semantic component atomic unit in the user behavior activity text semantic component atomic unit set to obtain the user behavior activity text semantic relationship between each text basic semantic component atomic unit; constructing a semantic relationship graph for each corresponding text basic semantic component atomic unit based on the user behavior activity text semantic relationship between each text basic semantic component atomic unit to generate a user behavior activity text semantic relationship graph; In an embodiment of the present invention, the semantic relationship between each text basic semantic component atomic unit in the previously extracted user behavior activity text semantic component atomic unit set is mined and analyzed to discover the potential semantic association between the text basic semantic component atomic units. The analysis method adopted is relationship mining based on semantic models, such as word embedding models such as Word2Vec or GloVe. The semantic relationship is extracted by calculating the vector space similarity between different text basic semantic component atomic units. In the specific operation, the semantic component atomic unit is first mapped to the vector space through a pre-trained word embedding model (such as Word2Vec or FastText), and then the cosine similarity between each text basic semantic component atomic unit is calculated to determine the strength of its semantic relationship. The implicit relationship between different semantic units in the text can be represented in a numerical way, thereby obtaining the user behavior activity text semantic relationship between each text basic semantic component atomic unit. At the same time, by combining the user behavior activity text semantic relations between the basic semantic component atomic units of each text obtained in the previous analysis, the corresponding text basic semantic component atomic units are connected and constructed into a semantic relationship graph, so that each basic semantic component atomic unit is regarded as a node in the graph, and the semantic relationship is the edge between the nodes. When constructing the graph, the weight of the edge is derived from the similarity of the semantic relationship mining results, that is, the edge weight between semantic component pairs with higher relationship strength is greater. This process adopts graph database technology for storage and management, and uses graph database systems such as Neo4j to convert text data and semantic relations into graph structures for modeling, ensuring that the semantic relationship structure of user behavior activities is clear and operational, and finally constructing a user behavior activity text semantic relationship graph.

[0027] Step S15: Based on the user behavior activity text semantic relationship graph, semantic feature recognition analysis is performed on the corresponding user behavior activity description text in the standardized user behavior activity text big data set to obtain a semantic feature set of the user behavior activity description text.

[0028] In an embodiment of the present invention, semantic features of the corresponding user behavior activity description text in the standardized user behavior activity text big data set are identified and analyzed by combining the semantic relationship graph of the user behavior activity text constructed previously, so as to group the text through the relationship graph stored in the graph database and the community detection algorithm in graph theory (such as Louvain algorithm or spectral clustering) to identify user groups with similar behaviors in a specific semantic field. In the feature extraction process, the nodes in the graph are classified by using a machine learning model (such as support vector machine SVM or random forest) to extract the main semantic features related to the user behavior activity. The feature set is further screened and optimized through feature engineering technology, so as to obtain a set of semantic feature sets that are helpful for further user behavior analysis and prediction, and finally obtain a semantic feature set of the user behavior activity description text.

[0029] Furthermore, the step S14 of constructing a semantic relationship graph for each corresponding text basic semantic component atomic unit based on the user behavior activity text semantic relationship between each text basic semantic component atomic unit includes the following steps: Perform text frequency and reverse document frequency statistics calculation on each text basic semantic component atomic unit in the user behavior activity text semantic component atomic unit set, so as to obtain the text frequency and reverse document frequency corresponding to each text basic semantic component atomic unit; In an embodiment of the present invention, by performing statistical calculations on the text frequency and the reverse document frequency for each basic semantic component atomic unit of the text in the previously extracted user behavior activity text semantic component atomic unit set, the text is segmented using a word segmentation tool in natural language processing technology (such as NLTK, SpaCy or Jieba) to extract each basic semantic component atomic unit. Then, by constructing a document-term matrix (Document-Term Matrix, DTM), the occurrence frequency TF (t, d) (Term Frequency, TF) of each atomic unit in the document is calculated, where t represents the occurrence time of the atomic unit in the document, and d is the number of occurrences of the atomic unit in the document. The reverse document frequency (Inverse Document Frequency, IDF) is calculated by calculating the number of documents containing a specific atomic unit, and then applying the formula ,in is the total number of documents, is the number of documents containing the atomic unit, and finally the text frequency and reverse document frequency corresponding to each atomic unit of the basic semantic component of the text are obtained.

[0030] Preferably, a semantic relationship weight distribution calculation is performed on the user behavior activity text semantic relationship between each text basic semantic component atomic unit based on the text frequency and the reverse document frequency corresponding to each text basic semantic component atomic unit, so as to obtain the text semantic relationship distribution weight between each text basic semantic component atomic unit; In an embodiment of the present invention, the user behavior activity text semantic relationship between each text basic semantic component atomic unit is further analyzed by combining the text frequency corresponding to each text basic semantic component atomic unit and the reverse document frequency calculated previously, so as to quantify the similarity between the atomic units by using the weighted cosine similarity or correlation coefficient method, thereby assigning semantic relationship weights. The specific operation includes constructing a similarity matrix, wherein the rows and columns of the matrix represent different basic semantic component atomic units respectively, and for each pair of atomic units, calculating the cosine similarity by their TF-IDF values ​​to obtain a similarity value between 0 and 1 as the semantic relationship weight between the pair of atomic units. The linear algebra module in the NumPy library can be used to perform matrix calculations, and these weights are integrated into a weight allocation list to form a semantic relationship mapping between each atomic unit, and finally the text semantic relationship allocation weights between each text basic semantic component atomic unit are obtained.

[0031] Preferably, the user behavior activity text semantic relationship between each text basic semantic component atomic unit is determined by semantic relationship weight mapping according to the text semantic relationship allocation weight between each text basic semantic component atomic unit, so as to obtain the user activity text semantic relationship weight mapping rule between each text basic semantic component atomic unit; In an embodiment of the present invention, the weights of the text semantic relations between the atomic units of the basic semantic components of the text obtained by the previous analysis are mapped and determined in combination with the text semantic relations between the atomic units of the basic semantic components of the text obtained by the previous assignment, so as to determine the relationship mapping rules between the atomic units of the basic semantic components in the user behavior activity text. By constructing a weighted directed graph, in which the nodes represent the atomic units of the basic semantic components and the weights of the edges represent the strength of the semantic relations between them, in a specific implementation, the Dijkstra algorithm in graph theory is used to calculate the shortest path, thereby revealing the core connection between the atomic units in the user behavior activity. At the same time, the relationship map is visualized using graphic visualization tools (such as NetworkX and Matplotlib), so that the relationship between the atomic units is clear at a glance. Finally, the mapping analysis obtains the weight mapping rules of the user activity text semantic relations between the atomic units of the basic semantic components of the text.

[0032] Preferably, based on the user activity text semantic relationship weight mapping rules between each text basic semantic component atomic unit, the user behavior activity text semantic relationship between each text basic semantic component atomic unit and each corresponding text basic semantic component atomic unit are constructed to generate a user behavior activity text semantic relationship map.

[0033] In an embodiment of the present invention, by combining the previously determined user activity text semantic relationship weight mapping rules between the basic semantic component atomic units of the text, the user behavior activity text semantic relationship between the basic semantic component atomic units of the text and each corresponding basic semantic component atomic unit are connected to construct a semantic relationship graph, so as to achieve in-depth analysis of each basic semantic component atomic unit and its semantic relationship, and to construct a multi-level semantic relationship graph with the basic semantic component atomic units of the text as nodes, the user behavior activity text semantic relationship as edges, and the corresponding user activity text semantic relationship weights are assigned to the edges. This graph not only shows the direct connection between the basic semantic component atomic units, but also includes indirect connections and hierarchical relationships. Graph database technology (such as Neo4j) can be used to store and manage these relationships, and the query language of the graph database (such as Cypher) can be used for efficient retrieval and analysis. In addition, the user behavior pattern is identified and classified in combination with a machine learning algorithm (such as cluster analysis or classification algorithm), so as to further optimize the structure and content of the semantic relationship graph, and finally construct a user behavior activity text semantic relationship graph.

[0034] Further, as an embodiment of the present invention, refer to Figure 3 As shown, Figure 1 Detailed step flow diagram of step S2 in FIG. 1 , in this embodiment, step S2 includes the following steps: Step S21: performing embedded feature selection processing on the semantic feature set of the user behavior activity description text to obtain a user behavior semantic embedded feature subset; In an embodiment of the present invention, by utilizing a pre-trained deep learning model, such as BERT or Word2Vec, the corresponding user behavior semantic sub-features in the semantic feature set of the user behavior activity description text are vectorized, and the user behavior semantic sub-features are converted into dense vector representations in a high-dimensional feature space. By analyzing these vectors, the relationship strength between each feature and the target variable can be calculated, and the L1 regularization method (Lasso regression) is used to perform feature selection, retaining features with a high correlation with user behavior performance, and eliminating redundant and irrelevant features. The generated user behavior semantic embedded feature subset includes sub-features that have a significant impact on user behavior activities, and finally a user behavior semantic embedded feature subset is obtained.

[0035] Step S22: performing filtering feature selection processing on the semantic feature set of the user behavior activity description text to obtain a user behavior semantic filtering feature subset; In an embodiment of the present invention, a filtering feature selection is performed on the corresponding user behavior semantic sub-features in the semantic feature set of the user behavior activity description text, so as to adopt statistical methods such as information gain, chi-square test or mutual information to evaluate the independence between each user behavior semantic sub-feature and the user behavior result, and by calculating the chi-square value of each user behavior semantic sub-feature, the features with chi-square values ​​higher than a preset threshold are screened out to form a user behavior semantic filtering feature subset. During the filtering process, the tools used, such as the feature selection module of Scikit-learn, can effectively help to quickly calculate the feature score and filter. The goal of this step is to obtain features with strong discrimination ability in different user behavior scenarios, and finally obtain the user behavior semantic filtering feature subset.

[0036] Step S23: performing repeated feature screening processing according to the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset to obtain a user behavior semantic repeated feature set; In an embodiment of the present invention, duplicate features are screened for the previously identified and screened user behavior semantic embedded feature subset and the filtered feature subset. At this time, the two feature subsets are subjected to an intersection operation to identify duplicate features. On this basis, those features that have obvious overlaps in user behavior semantics are further screened out. The Pandas library in Python can be used to perform data frame operations to efficiently process the intersection of feature sets and deduplicate them, and finally obtain a duplicate feature set of user behavior semantics.

[0037] Step S24: using the behavior semantic representativeness calculation formula to perform representativeness score calculation on each user behavior semantic sub-feature in the user behavior semantic repeated feature set, so as to obtain a feature representativeness score value corresponding to each user behavior semantic sub-feature; In an embodiment of the present invention, a suitable behavior semantic representativeness calculation formula is formed by combining the corresponding user behavior semantic sub-features in the user behavior semantic repetitive feature set, normalization constants, lower limit of integration time range, upper limit of integration time range, time variable parameters, weight parameters, exponential functions, feature center positions, feature dispersion, semantic feature similarity measurement values ​​and related parameters to perform representative score calculation on each user behavior semantic sub-feature in the user behavior semantic repetitive feature set, so as to ensure that the representative score of each sub-feature is accurately calculated, and finally obtain the feature representativeness score value corresponding to each user behavior semantic sub-feature.

[0038] The specific calculation formula for the representativeness of behavioral semantics is: ; In the formula, For the User behavior semantic sub-features The corresponding feature representative score value, For the User behavior semantic sub-features, For the User behavior semantic sub-features, and are all item index parameters of user behavior semantic sub-features, is the total number of semantic sub-features of user behavior, is the normalization constant, is the lower limit of the integration time range, is the upper limit of the integration time range, is the time variable parameter, For the The weight parameter corresponding to the semantic sub-feature of user behavior, is an exponential function, For the The feature center position corresponding to each user behavior semantic sub-feature in the user behavior semantic repeated feature set, For the The degree of feature dispersion corresponding to each user behavior semantic sub-feature in the user behavior semantic repeated feature set, For the The semantic sub-features of user behavior and User behavior semantic sub-features at time point The semantic feature similarity measure at is the correction factor of the characteristic representativeness score; The present invention obtains a behavior semantic representative calculation formula by using a specific mathematical model and after verification, which is used to calculate the representative score of each user behavior semantic sub-feature in the user behavior semantic repeated feature set. The behavior semantic representative calculation formula can ensure that the scores of all features are in the same range by using a normalization constant, thereby avoiding the order of magnitude difference between features affecting the final result. This process helps to improve the comparability of the score, making the feature selection process more stable and accurate. By using the weight parameters of the user behavior semantic sub-features, it is allowed to weight the score according to the importance of the feature in different contexts. By setting different weights, the importance of the feature can be flexibly adjusted, so as to better adapt to the diversity and complexity of user behavior. At the same time, by considering the central position and dispersion of the feature, the formula can reflect the clustering of the feature and its position in the overall feature space, which makes it possible to more accurately evaluate the similarity between features, thereby avoiding the selection of redundant features and improving the simplicity and effectiveness of the feature set. Considering the similarity between features through the semantic feature similarity metric can help identify similar features, thereby reducing redundant features. This mechanism helps to improve the efficiency of feature selection, ensure that only features with unique information are retained, and further enhance the expressiveness of the model. By introducing the time variable, the dynamic changes of user behavior can be analyzed, which is crucial for capturing the temporal characteristics of user behavior, so that the model can better adapt to behavioral changes in different time periods, thereby improving the accuracy of predictions. In addition, the introduction of the correction coefficient can fine-tune the feature score. Taking into account the uncertainty or other potential factors in the calculation process, this adjustment can improve the reliability of the score, making the final comprehensive representative feature set more in line with the actual situation. This formula can effectively identify and evaluate the representativeness of the semantic features of user behavior by comprehensively considering multiple factors such as feature weights, central positions, dispersion, similarities, and time changes. This mechanism not only improves the accuracy and effectiveness of feature selection, but also enhances the predictive ability and reliability of the user behavior analysis model, thereby achieving a deeper understanding of user behavior and providing strong support for personalized services and optimized decisions. In summary, this formula fully takes into account the first User behavior semantic sub-features Corresponding feature representative score value , No. User behavior semantic sub-features , No. User behavior semantic sub-features , item index parameter of user behavior semantic sub-feature and , the total number of semantic sub-features of user behavior , the normalization constant , lower limit of integration time range , upper limit of integration time range , time variable parameter , No. The weight parameter corresponding to the semantic sub-feature of user behavior , exponential function , No. The feature center position corresponding to each user behavior semantic sub-feature in the user behavior semantic repeated feature set , No. The degree of feature dispersion corresponding to each user behavior semantic sub-feature in the user behavior semantic repeated feature set , No. The semantic sub-features of user behavior and User behavior semantic sub-features at time point The semantic feature similarity measure at , correction factor for the feature representativeness score , according to User behavior semantic sub-features Corresponding feature representative score value The correlation between the above parameters constitutes a functional relationship This formula can realize the representative score calculation process for each user behavior semantic sub-feature in the user behavior semantic repeated feature set. At the same time, the correction coefficient of the feature representative score value is The introduction of can be adjusted according to the errors that occur in the calculation process, thereby improving the accuracy and applicability of the behavioral semantic representative calculation formula.

[0039] Step S25: Compare and judge the feature representativeness score value corresponding to each user behavior semantic sub-feature according to the preset representativeness score threshold; when the feature representativeness score value is greater than or equal to the preset representativeness score threshold, the corresponding user behavior semantic sub-feature is marked as a representative feature; when the feature representativeness score value is less than the preset representativeness score threshold, the corresponding user behavior semantic sub-feature is marked as a non-representative feature; the user behavior semantic sub-features marked as representative features in the user behavior semantic repeated feature set are screened out and combined to obtain a comprehensive representative feature set of user behavior semantics.

[0040] In the embodiment of the present invention, the feature representative score values ​​corresponding to the user behavior semantic sub-features obtained by the previous quantitative calculation are compared and judged by using a preset representative score threshold to set a reasonable score threshold. The value is based on the analysis results of the feature performance in the previous step to ensure that it can effectively distinguish representative features from non-representative features. Subsequently, the feature representative score values ​​corresponding to each user behavior semantic sub-feature are traversed, and the representative features (if the feature representative score value is greater than or equal to the preset representative score threshold) and non-representative features (if the feature representative score value is less than the preset representative score threshold) are marked through conditional judgment statements. At the same time, the user behavior semantic sub-features marked as representative features are screened out and integrated into a data set, and finally a user behavior semantic comprehensive representative feature set is obtained.

[0041] Further, step S21 includes the following steps: Perform feature importance evaluation and analysis on each user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain the feature importance degree corresponding to each user behavior semantic sub-feature; In an embodiment of the present invention, an importance evaluation analysis is performed on each user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text obtained by the previous analysis, so as to calculate the information gain or the reduction of Gini impurity of each feature in the tree model by adopting an integrated learning algorithm such as random forest and gradient boosting tree, and construct multiple decision trees. The feature importance of each user behavior semantic sub-feature can be determined by calculating the frequency of feature selection and its contribution to the model prediction results, thereby quantitatively calculating the feature importance degree of each sub-feature, and finally obtaining the feature importance degree corresponding to each user behavior semantic sub-feature.

[0042] Preferably, a feature sparsity quantification calculation is performed on each user behavior semantic sub-feature in the user behavior activity description text semantic feature set to obtain a feature sparsity degree corresponding to each user behavior semantic sub-feature; In an embodiment of the present invention, the feature sparsity of each user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text obtained by previous analysis is quantitatively calculated, so as to analyze the sparsity of each feature in the user behavior activity description text by using the Lasso regression method. In the specific implementation, an appropriate regularization parameter is selected, and the feature coefficient is limited by minimizing the objective function of the model, so as to quantitatively evaluate the sparsity of the feature. The sparsity can be achieved by calculating the ratio of the number of non-zero feature coefficients to the total number of features, and finally the feature sparsity corresponding to each user behavior semantic sub-feature is obtained.

[0043] Preferably, L1 regularization statistical analysis is performed on each user behavior semantic sub-feature corresponding to the user behavior activity description text semantic feature set according to the feature importance degree and feature sparsity degree corresponding to each user behavior semantic sub-feature, to obtain the user behavior semantic feature L1 regularization strength parameter; In an embodiment of the present invention, a statistical analysis of the L1 regularization strength is performed on the corresponding user behavior semantic sub-features in the semantic feature set of the user behavior activity description text by combining the feature importance and feature sparsity corresponding to the user behavior semantic sub-features obtained by previous quantitative calculation. This step aims to improve the effect of feature selection by constructing a comprehensive model including feature importance and sparsity in the form of L1 regularization. During implementation, the weights of feature importance and sparsity are first determined, and significant user behavior semantic sub-features are selected by setting appropriate thresholds. Then, the contribution of the feature is adjusted using the regularization parameter, and a linear regression analysis is performed, so as to statistically calculate the corresponding L1 regularization strength parameter, and finally the L1 regularization strength parameter of the user behavior semantic feature is obtained.

[0044] Preferably, based on the L1 regularization strength parameter of the user behavior semantic feature, L1 regularized embedded feature selection is performed on each corresponding user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain a user behavior semantic embedded feature subset.

[0045] In an embodiment of the present invention, L1 regularized embedded feature selection is performed on each corresponding user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text by combining the L1 regularization strength parameter of the user behavior semantic feature obtained by previous quantitative calculation. In a specific implementation, a Lasso regression model is used to process the semantic feature set of the user behavior activity description text through the corresponding L1 regularization strength parameter of the user behavior semantic feature. During the model training process, the coefficient of the feature is gradually shrunk to zero, thereby achieving feature selection and elimination. By setting a threshold, features that are strongly correlated with user behavior can be effectively screened out, and a subset containing important user behavior semantic features can be formed, and finally a user behavior semantic embedded feature subset is selected.

[0046] Further, step S22 includes the following steps: Performing statistical calculations on the number of occurrences of activity descriptions for each user behavior semantic sub-feature in the user behavior activity description text semantic feature set, and obtaining the number of occurrences of user behavior activity descriptions corresponding to each user behavior semantic sub-feature; In an embodiment of the present invention, each corresponding user behavior semantic sub-feature is extracted from a semantic feature set of a user behavior activity description text, and a natural language processing (NLP) tool such as NLTK or SpaCy is used to pre-process the user behavior semantic sub-features in the corresponding user behavior activity description text, including word segmentation, stop word removal, and morphological restoration. Then, a statistical analysis library (such as Pandas) is used to count the frequency of occurrence of each user behavior semantic sub-feature in the text, and the number of occurrences of each sub-feature is recorded through a dictionary or set structure to ensure that all data is accurately recorded and organized, and finally the number of occurrences of the user behavior activity description corresponding to each user behavior semantic sub-feature is obtained.

[0047] Preferably, a feature frequency statistical analysis is performed on each corresponding user behavior semantic sub-feature according to the number of occurrences of the user behavior activity description corresponding to each user behavior semantic sub-feature, so as to obtain the user behavior semantic feature frequency corresponding to each user behavior semantic sub-feature; In an embodiment of the present invention, a statistical calculation of the feature frequency is performed on each corresponding user behavior semantic sub-feature by combining the number of occurrences of the user behavior activity description corresponding to each user behavior semantic sub-feature obtained by previous statistical analysis, and this is achieved by dividing the number of occurrences of each sub-feature by the total text length (or the total number of occurrences). Mathematical calculations are performed using numerical computing libraries such as NumPy or Pandas, and ultimately the user behavior semantic feature frequency corresponding to each user behavior semantic sub-feature is obtained.

[0048] Preferably, a feature correlation evaluation analysis is performed on each user behavior semantic sub-feature in the user behavior activity description text semantic feature set to obtain a correlation coefficient between each user behavior semantic sub-feature and the user behavior activity description text; a user behavior activity significance level is set according to the correlation coefficient between each user behavior semantic sub-feature and the user behavior activity description text to obtain a user behavior activity semantic feature significance level; In an embodiment of the present invention, the feature correlation coefficient between each user behavior semantic sub-feature and the user behavior activity description text in the semantic feature set of the user behavior activity description text obtained by the previous analysis is statistically calculated by using the Pearson correlation coefficient or the Spearman rank correlation coefficient, so as to calculate the correlation coefficient between all user behavior semantic sub-features and the text content through the SciPy library, and display the correlation through a visualization tool (such as Matplotlib or Seaborn), thereby obtaining the correlation coefficient between each user behavior semantic sub-feature and the user behavior activity description text. At the same time, by performing a significance test based on the correlation coefficient between the user behavior semantic sub-feature and the user behavior activity description text obtained by the previous quantitative calculation, the feature significance is evaluated by using a t-test or ANOVA analysis, thereby determining the corresponding significance level (for example, 0.05), and finally obtaining the significance level of the user behavior activity semantic feature.

[0049] Preferably, a sub-feature chi-square test is performed on each corresponding user behavior semantic sub-feature based on the significance level of the user behavior activity semantic feature and the user behavior semantic feature frequency corresponding to each user behavior semantic sub-feature to obtain a semantic feature chi-square value corresponding to each user behavior semantic sub-feature; In an embodiment of the present invention, a chi-square test of the sub-features is performed on the corresponding user behavior semantic sub-features by combining the significance level of the semantic features of the user behavior activity obtained by the previous analysis and the user behavior semantic feature frequency corresponding to the user behavior semantic sub-features, so as to construct a contingency table of the chi-square test, record the observed frequency and the expected frequency of each sub-feature, and use the chi2_contingency function in SciPy to perform the chi-square test and calculate the corresponding chi-square value. This process ensures that each user behavior semantic sub-feature undergoes a strict statistical test, and the obtained chi-square value will be recorded, and finally the semantic feature chi-square value corresponding to each user behavior semantic sub-feature is obtained.

[0050] Preferably, based on the semantic feature chi-square value corresponding to each user behavior semantic sub-feature, a filtering feature selection process is performed on each corresponding user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain a user behavior semantic filtering feature subset.

[0051] In an embodiment of the present invention, a filtered feature selection is performed on each corresponding user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text by combining the semantic feature chi-square value corresponding to each user behavior semantic sub-feature obtained by previous quantitative calculation, so as to set a chi-square threshold in advance. If the chi-square value is greater than or equal to the set chi-square threshold, the corresponding user behavior semantic sub-feature is filtered out to obtain sub-features with higher significance. This process uses Pandas to perform filtering operations on the data frame, and finally obtains the user behavior semantic filtered feature subset.

[0052] Further, step S3 includes the following steps: Step S31: extracting context information from each user behavior semantics representative sub-feature in the user behavior semantics comprehensive representative feature set to obtain user behavior semantics context information corresponding to each user behavior semantics representative sub-feature; In an embodiment of the present invention, context information is extracted from each user behavior semantic representative sub-feature in the user behavior semantic comprehensive representative feature set obtained by previous analysis. These sub-features include user clicks, search terms, purchase records, etc., and natural language processing (NLP) technology is used to extract context information related to these sub-features using methods such as word segmentation, part-of-speech tagging and named entity recognition. For example, for "the user clicked on the link of product A", keywords such as "user", "click", "product A" and "link" can be extracted, and then context information about the behavior is generated, including time, place, device type and other information. This process uses tools such as spaCy or NLTK, which can efficiently process text data and extract meaningful information, and finally extract the user behavior semantic context information corresponding to each user behavior semantic representative sub-feature.

[0053] Step S32: extracting sub-feature activity attributes from each corresponding user behavior semantic representative sub-feature based on the user behavior semantic context information corresponding to each user behavior semantic representative sub-feature, and obtaining the user behavior activity attributes corresponding to each user behavior semantic representative sub-feature; In an embodiment of the present invention, the sub-feature activity attributes of each corresponding user behavior semantic representative sub-feature are extracted by combining the user behavior semantic context information corresponding to each user behavior semantic representative sub-feature previously extracted. Taking the behavior of "user clicking on the link of product A" as an example, the activity attributes may include "product type", "click frequency", "user group", etc., and cluster analysis and classification algorithms (such as K-means or decision tree) are used to analyze the extracted context information to summarize the activity attributes of each sub-feature. By comparing historical data and combining user portraits, it is ensured that the extracted activity attributes are representative and reliable. This process can also be achieved through tools such as Scikit-learn or TensorFlow to implement data mining and feature engineering, and finally obtain the user behavior activity attributes corresponding to each user behavior semantic representative sub-feature.

[0054] Step S33: classifying each corresponding user behavior semantics representative sub-feature in the user behavior semantics comprehensive representative feature set based on the user behavior activity attribute corresponding to each user behavior semantics representative sub-feature, so as to obtain a behavior semantics representative feature subset corresponding to each user behavior activity; In an embodiment of the present invention, the behavior activity characteristics of each user behavior semantic representative sub-feature corresponding to the user behavior semantic comprehensive representative feature set are classified by combining the user behavior activity attributes corresponding to each user behavior semantic representative sub-feature obtained by previous mining and analysis. In this step, the user behavior activities can be classified with the help of a machine learning model (such as a support vector machine SVM or a random forest). The activity attributes of each sub-feature are used as input features. After the model is trained, it can identify the categories of different user behavior activities, such as "browsing", "purchasing", "commenting", etc. To ensure the accuracy of the classification, a cross-validation method is used to evaluate the performance of the model, and indicators such as accuracy, recall rate and F1 score are used to optimize the model. Finally, the behavior semantic representative feature subsets corresponding to each user behavior activity are classified.

[0055] Step S34: performing local interpretability evaluation and analysis on the behavioral semantic representative feature subset corresponding to each user behavior activity to obtain a local interpretability factor of the user behavior decision corresponding to each user behavior activity; In an embodiment of the present invention, a local interpretability evaluation and analysis is performed on each user behavior activity and its corresponding subset of behavioral semantic representative features. In this process, a local interpretable model-agnostic interpretation method (such as LIME or SHAP) is used to identify key factors affecting user decisions, so as to generate a local model and analyze the relationship between the user behavior activity characteristics and the decision results, thereby providing an interpretability factor for each decision. For example, if a user chooses to buy a product, LIME can help identify the role of features such as "discounts" and "product reviews" in the decision. This process needs to be combined with data visualization tools (such as Matplotlib or Tableau) to display the interpretability analysis results, so that the local interpretability factor of the decision is clear at a glance, that is, the degree of influence of the corresponding local influencing factors in the user decision-making process, and finally the local interpretability factor of the user behavior decision corresponding to each user behavior activity is obtained.

[0056] Step S35: Perform AI-driven intelligent parsing analysis on the local explainability factors of user behavior decisions corresponding to each user behavior activity to obtain the parsing results of the explainability factors of user behavior decisions corresponding to each user behavior activity.

[0057] In an embodiment of the present invention, AI-driven intelligent analysis is performed on the local interpretability factors of user behavior decisions corresponding to each user behavior activity previously obtained after evaluation and analysis. This step will combine deep learning technology (such as deep neural network) for factor analysis, so as to perform deeper feature learning of local interpretability factors by utilizing neural network models, and perform nonlinear mapping of the relationship between factors through the model to extract more complex decision logic. After the model is trained, the corresponding decision interpretability factor analysis results can be generated. For example, the model will point out that a certain user is more influenced by "social media influence" and "similar product reviews" when making a purchase. This process relies on deep learning frameworks (such as TensorFlow or PyTorch) and analysis tools to ensure that the final analysis results are highly credible and practical, and finally the user behavior decision interpretability factor analysis results corresponding to each user behavior activity are obtained.

[0058] Further, step S34 includes the following steps: Step S341: performing user behavior decision semantic association analysis on each behavior semantic representative sub-feature in the behavior semantic representative feature subset corresponding to each user behavior activity, to obtain the semantic association relationship between the behavior semantic representative sub-feature corresponding to each user behavior activity and the user behavior decision; In an embodiment of the present invention, a semantic association analysis of behavioral decisions is performed on each behavioral semantic representative sub-feature in a subset of behavioral semantic representative features corresponding to each user behavior activity previously screened to extract user behavior data, including browsing history, purchase records, search queries, etc., and these user behavior data are converted into behavioral semantic features by using natural language processing (NLP) technology. For example, the potential intention behind the user behavior is identified through keyword extraction and sentiment analysis, and statistical methods, such as correlation analysis or regression analysis, are applied to explore the association between behavioral semantic features and user decisions. This process can be implemented using the Python programming language and its related data analysis libraries (such as Pandas and Scikit-learn), and ultimately the semantic association between the behavioral semantic representative sub-features corresponding to each user behavior activity and the user behavior decision is obtained.

[0059] Step S342: Based on the semantic association relationship between the behavioral semantic representative sub-features corresponding to each user behavior activity and the user behavior decision, a behavior decision local influencing factor analysis is performed on the behavior semantic representative feature subset corresponding to each user behavior activity to obtain a user behavior decision local influencing factor candidate set corresponding to each user behavior activity; In an embodiment of the present invention, by combining the semantic association relationship between the behavioral semantic representative sub-features corresponding to each user behavior activity obtained in the previous analysis and the user behavior decision, the corresponding behavioral semantic representative feature subset is identified and analyzed for the local influencing factors of the behavior decision, so as to analyze the influence of each behavioral feature on the decision result by constructing a decision tree model or using SHAP (SHapleyAdditive exPlanations) value and other technologies. At this time, the selection of tools may include the Shap library in R language or Python to provide explainable results. In the specific operation, the user behavior activity data set is first divided into a training set and a test set to train the model and evaluate the local influencing factors of each feature, thereby forming a corresponding local influencing factor candidate set, and finally obtaining a user behavior decision local influencing factor candidate set corresponding to each user behavior activity.

[0060] Step S343: performing user behavior performance evaluation and analysis on each behavior semantics representative sub-feature in the behavior semantics representative feature subset corresponding to each user behavior activity, and obtaining a user behavior activity performance score corresponding to each behavior semantics representative sub-feature of each user behavior activity; In an embodiment of the present invention, an evaluation and analysis of user behavior performance is performed on each behavioral semantic representative sub-feature in a subset of behavioral semantic representative features corresponding to each user behavior activity previously screened, so as to adopt multi-dimensional evaluation indicators, such as conversion rate, user retention rate, and user satisfaction, and by setting a series of standards, these indicators are applied to different user behavior activities. For example, A / B testing is used to evaluate the impact of different strategies on user behavior performance. At the same time, statistical methods, such as analysis of variance (ANOVA) or t-test, are applied to compare the behavioral performance scores of different user groups to reflect the effectiveness and influence of each behavioral semantic representative sub-feature, and finally the user behavior activity performance score corresponding to each behavioral semantic representative sub-feature of each user behavior activity is obtained.

[0061] Step S344: Based on the user behavior activity performance score corresponding to each behavior semantic representative sub-feature of each user behavior activity, a local interpretability framework design is performed on the behavior semantic representative feature subset corresponding to each user behavior activity to generate a local interpretability evaluation and analysis model corresponding to each user behavior activity; In an embodiment of the present invention, a local interpretability model framework is designed for a corresponding subset of behavioral semantic representative features by combining the user behavior activity performance scores of each behavioral semantic representative sub-feature corresponding to each user behavior activity obtained by previous evaluation and calculation, so as to integrate interpretability tools such as LIME (Local Interpretable Model-agnostic Explanations) or SHAP, and perform interpretability evaluation on the subset of behavioral semantic representative features corresponding to each user behavior activity in combination with the corresponding user behavior activity performance scores, so as to build a local model, which is fitted around each user behavior activity to better understand the impact of features on decisions, and to display the contribution of each feature, that is, the user behavior activity performance score, by using graphical tools (such as Matplotlib or Seaborn) to facilitate analysis and interpretation. This framework will generate a corresponding local interpretability evaluation analysis model, aiming to make the decision-making process transparent and easy to understand, and finally design and generate a local interpretability evaluation analysis model corresponding to each user behavior activity.

[0062] Step S345: Perform local interpretability evaluation and analysis on the candidate set of local influencing factors of user behavior decisions corresponding to each user behavior activity according to the local interpretability evaluation and analysis model corresponding to each user behavior activity, and obtain the local interpretability factor of user behavior decision corresponding to each user behavior activity.

[0063] In an embodiment of the present invention, a local interpretability evaluation and analysis model corresponding to each user behavior activity generated in combination with a previously designed one is used to perform a local interpretability evaluation and analysis on the corresponding candidate set of local influencing factors of user behavior decisions, so as to use the above-mentioned LIME or SHAP method within the local interpretability framework to quantitatively analyze each local influencing factor in the candidate set to determine its role and importance in different user behavior activities. Specifically, regression analysis is used to calibrate the prediction results of the local model, and the performance of each local influencing factor in influencing the user decision-making process is further analyzed. In terms of operation, relevant data visualization tools in Python are used to generate a factor importance ranking graph to intuitively show the contribution of each local influencing factor to the user behavior decision, and finally the local interpretability factor of the user behavior decision corresponding to each user behavior activity is obtained.

[0064] Further, step S4 includes the following steps: Step S41: performing factor correlation statistical analysis on the corresponding user behavior decision interpretability factors in the user behavior decision interpretability factor analysis results corresponding to each user behavior activity, and obtaining the correlation between the decision interpretability factors corresponding to each user behavior activity; In an embodiment of the present invention, the Pearson correlation coefficient or the Spearman rank correlation coefficient is used to perform a statistical analysis of the correlation of the user behavior decision interpretability factors corresponding to each user behavior activity in the analysis results of the user behavior decision interpretability factors, so that the collected user behavior decision interpretability factors are input into statistical software or data analysis tools, such as the pandas library in Python or the psych package in the R language, and the correlation calculation function is used for calculation to obtain a correlation matrix between the factors, which clearly shows the strength and direction between the decision interpretability factors corresponding to each user behavior activity. Furthermore, a visualization tool (such as Matplotlib or Tableau) is used to display the correlation matrix in the form of a heat map, which is convenient for observing the potential relationship and mutual influence between the factors, and finally the correlation between the decision interpretability factors corresponding to each user behavior activity is obtained.

[0065] Step S42: constructing a decision logic relationship network for the corresponding user behavior decision interpretability factors based on the correlation between the decision interpretability factors corresponding to each user behavior activity, so as to generate a user behavior decision logic relationship network structure corresponding to each user behavior activity; In an embodiment of the present invention, a logical relationship network is constructed to connect the corresponding user behavior decision interpretability factors by combining the correlation between the decision interpretability factors corresponding to each user behavior activity obtained by previous quantitative calculation, so as to utilize the concepts of nodes and edges in graph theory, regard the user behavior decision interpretability factors as nodes, and the correlation between the factors as the weight of the edge, and with the help of network analysis tools such as Gephi or NetworkX, establish a directed graph containing all relevant factors, and set the weight of the edge to reflect the strength of its correlation, then apply a network analysis algorithm (such as Dijkstra algorithm or PageRank algorithm) to identify important factors and their mutual relationships, and generate the topological structure of the decision logic relationship network, which not only reflects the direct relationship between the factors, but also reveals the potential influence path of the factors in the decision-making process, and finally constructs a user behavior decision logic relationship network structure corresponding to each user behavior activity.

[0066] Step S43: Based on the user behavior decision logical relationship network structure corresponding to each user behavior activity, the user behavior decision interpretability factor analysis results corresponding to each user behavior activity are visualized and displayed with logical reasons to obtain the interpretable logical reasons behind each user behavior activity decision.

[0067] In an embodiment of the present invention, by combining the user behavior decision logic relationship network structure corresponding to each user behavior activity obtained by previous connection construction, the corresponding user behavior decision explainability factor analysis results are visualized for logical reasons, so as to more intuitively understand the decision logic reasons behind the user behavior activities. By selecting key factors based on the constructed decision logic relationship network and presenting their data in a graphical manner, interactive visualization graphics can be created using chart tools (such as D3.js or Plotly) to display the factors and the relationship and influence between them. By setting interactive functions, users can click on a factor to view its specific impact and logical path on the decision, and then analyze the importance of the factor in the decision-making process and its logical explanation reasons, thereby generating a corresponding decision logic display report, which will include a detailed analysis of the relationship between factors, an impact assessment and a comprehensive explanation of user behavior decisions, and ultimately obtain the explainable logical reasons behind the decisions of each user behavior activity.

[0068] Furthermore, the present invention also provides an AI-driven big data intelligent parsing and analysis system for executing the AI-driven big data intelligent parsing and analysis method as described above. The AI-driven big data intelligent parsing and analysis system includes: A user behavior semantic feature analysis module is used to obtain a large data set of user behavior activity description texts, and perform behavior text semantic feature analysis on the large data set of user behavior activity description texts, thereby obtaining a user behavior activity description text semantic feature set; The user behavior semantic repeated representative screening module is used to perform embedded and filtered feature selection processing on the semantic feature set of the user behavior activity description text to obtain the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset; perform repeated representative feature screening processing based on the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset, so as to obtain the user behavior semantic comprehensive representative feature set; The behavior activity AI-driven intelligent analysis module is used to classify the behavior activity features of the user behavior semantics comprehensive representative feature set to obtain the behavior semantics representative feature subset corresponding to each user behavior activity; perform local interpretable AI intelligent analysis on the behavior semantics representative feature subset corresponding to each user behavior activity to obtain the user behavior decision interpretability factor analysis result corresponding to each user behavior activity; The behavior decision analysis reason visualization module is used to visualize the logical reasons of the user behavior decision explainability factor analysis results corresponding to each user behavior activity, so as to obtain the explainable logical reasons behind each user behavior activity decision.

[0069] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A big data intelligent parsing and analysis method based on AI, characterized in that: The following steps are involved: Step S1: obtaining a large dataset of user behavior activity description texts, and performing behavior text semantic feature analysis on the large dataset of user behavior activity description texts to obtain a user behavior activity description text semantic feature set; Step S2: performing embedded and filtered feature selection processing on the semantic feature set of the user behavior activity description text to obtain a user behavior semantic embedded feature subset and a user behavior semantic filtered feature subset; performing repeated representative feature screening processing on the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset to obtain a user behavior semantic comprehensive representative feature set; Step S3: classifying the behavior activity features of the user behavior semantics comprehensive representative feature set to obtain a behavior semantics representative feature subset corresponding to each user behavior activity; Perform local interpretable AI intelligent analysis on the behavioral semantic representative feature subsets corresponding to each user behavior activity to obtain the interpretable factor analysis results of user behavior decisions corresponding to each user behavior activity; Step S4: Perform a logical reason visualization processing on the user behavior decision explainability factor analysis results corresponding to each user behavior activity to obtain the explainable logical reasons behind each user behavior activity decision.

2. The AI-driven big data intelligent parsing and analysis method according to claim 1 is characterized in that: Step S1 includes the following steps: Step S11: obtaining a large data set of user behavior activity description texts, wherein the large data set of user behavior activity description texts includes user behavior activity text record data from user behavior data sources related to social media, mobile applications, online platforms, and e-commerce websites; Step S12: performing sensitivity elimination processing on the user behavior activity description text big data set to obtain the user behavior activity text sensitive word elimination big data set; performing stop word removal and text normalization processing on the user behavior activity text sensitive word elimination big data set to obtain the standardized user behavior activity text big data set; Step S13: extracting semantic atomic units from the standardized user behavior activity text big data set to obtain a set of semantic component atomic units of the user behavior activity text; Step S14: performing text semantic relationship mining and analysis on each text basic semantic component atomic unit in the user behavior activity text semantic component atomic unit set to obtain the user behavior activity text semantic relationship between each text basic semantic component atomic unit; constructing a semantic relationship graph for each corresponding text basic semantic component atomic unit based on the user behavior activity text semantic relationship between each text basic semantic component atomic unit to generate a user behavior activity text semantic relationship graph; Step S15: Based on the user behavior activity text semantic relationship graph, semantic feature recognition analysis is performed on the corresponding user behavior activity description text in the standardized user behavior activity text big data set to obtain a semantic feature set of the user behavior activity description text.

3. The AI-driven big data intelligent parsing and analysis method according to claim 2 is characterized in that: The step S14 of constructing a semantic relationship graph for each corresponding text basic semantic component atomic unit based on the user behavior activity text semantic relationship between each text basic semantic component atomic unit includes the following steps: Perform text frequency and reverse document frequency statistics calculation on each text basic semantic component atomic unit in the user behavior activity text semantic component atomic unit set, so as to obtain the text frequency and reverse document frequency corresponding to each text basic semantic component atomic unit; Based on the text frequency and reverse document frequency corresponding to each text basic semantic component atomic unit, the semantic relationship weight distribution calculation of the user behavior activity text semantic relationship between each text basic semantic component atomic unit is performed to obtain the text semantic relationship distribution weight between each text basic semantic component atomic unit; According to the text semantic relationship allocation weights between the basic semantic component atomic units of each text, the user behavior activity text semantic relationship between the basic semantic component atomic units of each text is determined by semantic relationship weight mapping, so as to obtain the user activity text semantic relationship weight mapping rules between the basic semantic component atomic units of each text; Based on the user activity text semantic relationship weight mapping rules between each text basic semantic component atomic unit, the user behavior activity text semantic relationship between each text basic semantic component atomic unit and each corresponding text basic semantic component atomic unit are constructed to generate a user behavior activity text semantic relationship map.

4. The AI-driven big data intelligent parsing and analysis method according to claim 1 is characterized in that: Step S2 includes the following steps: Step S21: performing embedded feature selection processing on the semantic feature set of the user behavior activity description text to obtain a user behavior semantic embedded feature subset; Step S22: performing filtering feature selection processing on the semantic feature set of the user behavior activity description text to obtain a user behavior semantic filtering feature subset; Step S23: performing repeated feature screening processing according to the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset to obtain a user behavior semantic repeated feature set; Step S24: using the behavior semantic representativeness calculation formula to perform representativeness score calculation on each user behavior semantic sub-feature in the user behavior semantic repeated feature set, so as to obtain a feature representativeness score value corresponding to each user behavior semantic sub-feature; The specific calculation formula for the representativeness of behavioral semantics is: ； In the formula, For the User behavior semantic sub-features The corresponding feature representative score value, For the User behavior semantic sub-features, For the User behavior semantic sub-features, and are all item index parameters of user behavior semantic sub-features, is the total number of semantic sub-features of user behavior, is the normalization constant, is the lower limit of the integration time range, is the upper limit of the integration time range, is the time variable parameter, For the The weight parameter corresponding to the semantic sub-feature of user behavior, is an exponential function, For the The feature center position corresponding to each user behavior semantic sub-feature in the user behavior semantic repeated feature set, For the The degree of feature dispersion corresponding to each user behavior semantic sub-feature in the user behavior semantic repeated feature set, For the The semantic sub-features of user behavior and User behavior semantic sub-features at time point The semantic feature similarity measure at is the correction factor of the characteristic representativeness score; Step S25: Compare and judge the feature representativeness score value corresponding to each user behavior semantic sub-feature according to the preset representativeness score threshold; when the feature representativeness score value is greater than or equal to the preset representativeness score threshold, the corresponding user behavior semantic sub-feature is marked as a representative feature; when the feature representativeness score value is less than the preset representativeness score threshold, the corresponding user behavior semantic sub-feature is marked as a non-representative feature; the user behavior semantic sub-features marked as representative features in the user behavior semantic repeated feature set are screened out and combined to obtain a comprehensive representative feature set of user behavior semantics.

5. The AI-driven big data intelligent parsing and analysis method according to claim 4 is characterized in that: Step S21 includes the following steps: Perform feature importance evaluation and analysis on each user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain the feature importance degree corresponding to each user behavior semantic sub-feature; Perform feature sparsity quantification calculation on each user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain the feature sparsity degree corresponding to each user behavior semantic sub-feature; According to the feature importance and feature sparsity of each user behavior semantic sub-feature, L1 regularization statistical analysis is performed on each user behavior semantic sub-feature corresponding to the user behavior activity description text semantic feature set to obtain the user behavior semantic feature L1 regularization strength parameter; Based on the L1 regularization strength parameter of the user behavior semantic feature, L1 regularized embedded feature selection is performed on each corresponding user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain the user behavior semantic embedded feature subset.

6. The AI-driven big data intelligent parsing and analysis method according to claim 4 is characterized in that: Step S22 includes the following steps: Performing statistical calculations on the number of occurrences of activity descriptions for each user behavior semantic sub-feature in the user behavior activity description text semantic feature set, and obtaining the number of occurrences of user behavior activity descriptions corresponding to each user behavior semantic sub-feature; Perform feature frequency statistical analysis on each corresponding user behavior semantic sub-feature according to the number of occurrences of the user behavior activity description corresponding to each user behavior semantic sub-feature, and obtain the user behavior semantic feature frequency corresponding to each user behavior semantic sub-feature; Performing feature correlation evaluation and analysis on each user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain the correlation coefficient between each user behavior semantic sub-feature and the user behavior activity description text; setting the user behavior activity significance level according to the correlation coefficient between each user behavior semantic sub-feature and the user behavior activity description text to obtain the user behavior activity semantic feature significance level; Based on the significance level of the semantic feature of the user behavior activity and the user behavior semantic feature frequency corresponding to each user behavior semantic sub-feature, a sub-feature chi-square test is performed on each corresponding user behavior semantic sub-feature to obtain the semantic feature chi-square value corresponding to each user behavior semantic sub-feature; Based on the semantic feature chi-square value corresponding to each user behavior semantic sub-feature, a filtering feature selection process is performed on each corresponding user behavior semantic sub-feature in the semantic feature set of the user behavior activity description text to obtain a user behavior semantic filtering feature subset.

7. The AI-driven big data intelligent parsing and analysis method according to claim 1 is characterized in that: Step S3 includes the following steps: Step S31: extracting context information from each user behavior semantics representative sub-feature in the user behavior semantics comprehensive representative feature set to obtain user behavior semantics context information corresponding to each user behavior semantics representative sub-feature; Step S32: extracting sub-feature activity attributes from each corresponding user behavior semantic representative sub-feature based on the user behavior semantic context information corresponding to each user behavior semantic representative sub-feature, and obtaining the user behavior activity attributes corresponding to each user behavior semantic representative sub-feature; Step S33: classifying each corresponding user behavior semantics representative sub-feature in the user behavior semantics comprehensive representative feature set based on the user behavior activity attribute corresponding to each user behavior semantics representative sub-feature, so as to obtain a behavior semantics representative feature subset corresponding to each user behavior activity; Step S34: performing local interpretability evaluation and analysis on the behavioral semantic representative feature subset corresponding to each user behavior activity to obtain a local interpretability factor of the user behavior decision corresponding to each user behavior activity; Step S35: Perform AI-driven intelligent parsing analysis on the local explainability factors of user behavior decisions corresponding to each user behavior activity to obtain the parsing results of the explainability factors of user behavior decisions corresponding to each user behavior activity.

8. The AI-driven big data intelligent parsing and analysis method according to claim 7 is characterized in that: Step S34 includes the following steps: Step S341: Performing user behavior decision semantic association analysis on each behavior semantic representative sub-feature in the behavior semantic representative feature subset corresponding to each user behavior activity, to obtain the semantic association relationship between the behavior semantic representative sub-feature corresponding to each user behavior activity and the user behavior decision; Step S342: Based on the semantic association relationship between the behavioral semantic representative sub-features corresponding to each user behavior activity and the user behavior decision, a behavior decision local influencing factor analysis is performed on the behavior semantic representative feature subset corresponding to each user behavior activity to obtain a user behavior decision local influencing factor candidate set corresponding to each user behavior activity; Step S343: performing user behavior performance evaluation and analysis on each behavior semantics representative sub-feature in the behavior semantics representative feature subset corresponding to each user behavior activity, and obtaining a user behavior activity performance score corresponding to each behavior semantics representative sub-feature of each user behavior activity; Step S344: Based on the user behavior activity performance score corresponding to each behavior semantic representative sub-feature of each user behavior activity, a local interpretability framework design is performed on the behavior semantic representative feature subset corresponding to each user behavior activity to generate a local interpretability evaluation and analysis model corresponding to each user behavior activity; Step S345: Perform local interpretability evaluation and analysis on the candidate set of local influencing factors of user behavior decisions corresponding to each user behavior activity according to the local interpretability evaluation and analysis model corresponding to each user behavior activity, and obtain the local interpretability factor of user behavior decision corresponding to each user behavior activity.

9. The AI-driven big data intelligent parsing and analysis method according to claim 1 is characterized in that: Step S4 includes the following steps: Step S41: performing factor correlation statistical analysis on the corresponding user behavior decision interpretability factors in the user behavior decision interpretability factor analysis results corresponding to each user behavior activity, and obtaining the correlation between the decision interpretability factors corresponding to each user behavior activity; Step S42: constructing a decision logic relationship network for the corresponding user behavior decision interpretability factors based on the correlation between the decision interpretability factors corresponding to each user behavior activity, so as to generate a user behavior decision logic relationship network structure corresponding to each user behavior activity; Step S43: Based on the user behavior decision logical relationship network structure corresponding to each user behavior activity, the user behavior decision interpretability factor analysis results corresponding to each user behavior activity are processed for logical reason visualization to obtain the interpretable logical reasons behind each user behavior activity decision.

10. An AI-driven big data intelligent parsing and analysis system, characterized in that: Used to execute the AI-driven big data intelligent parsing and analysis method as claimed in claim 1, the AI-driven big data intelligent parsing and analysis system comprises: A user behavior semantic feature analysis module is used to obtain a large data set of user behavior activity description texts, and perform behavior text semantic feature analysis on the large data set of user behavior activity description texts, thereby obtaining a user behavior activity description text semantic feature set; The user behavior semantic repeated representative screening module is used to perform embedded and filtered feature selection processing on the semantic feature set of the user behavior activity description text to obtain the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset; perform repeated representative feature screening processing based on the user behavior semantic embedded feature subset and the user behavior semantic filtered feature subset, so as to obtain the user behavior semantic comprehensive representative feature set; The behavior activity AI-driven intelligent analysis module is used to classify the behavior activity features of the user behavior semantics comprehensive representative feature set to obtain the behavior semantics representative feature subset corresponding to each user behavior activity; perform local interpretable AI intelligent analysis on the behavior semantics representative feature subset corresponding to each user behavior activity to obtain the user behavior decision interpretability factor analysis result corresponding to each user behavior activity; The behavior decision analysis reason visualization module is used to visualize the logical reasons of the user behavior decision explainability factor analysis results corresponding to each user behavior activity, so as to obtain the explainable logical reasons behind each user behavior activity decision.

Images