User behavior intelligent analysis and management system based on big data technology

Abstract

The invention relates to the technical field of user behavior analysis, and discloses a user behavior intelligent analysis and management system based on a big data technology. The system comprises a user behavior data acquisition module for acquiring behavior data in a multi-dimensional scene; the behavior feature intelligent recognition module is used for extracting features by using a deep neural network and a time sequence analysis algorithm and generating a map; the behavior pattern dynamic analysis module is used for analyzing pattern changes through dynamic clustering and a hidden Markov model; the abnormal behavior autonomous detection module is used for detecting anomalies based on the multi-dimensional anomaly score and an adaptive threshold value; and the behavior management intelligent optimization module is used for optimizing a management strategy by utilizing a reinforcement learning algorithm. In addition, a behavior data archiving module is further arranged to guarantee safe storage of data. The system can comprehensively collect and analyze user behavior data, accurately detect abnormity, intelligently optimize a management strategy, improve user experience, system safety and operation efficiency, and have wide application value in multiple fields.

Description

Technical Field

[0001] The present invention relates to the technical field of user behavior analysis, and specifically to a user behavior intelligent analysis and management system based on big data technology. Background Art

[0002] In today's digital age, a vast amount of data is continuously generated, and the behavior data of users in various systems contains rich value. However, there are many deficiencies in the existing means of user behavior analysis and management, making it difficult to fully exploit the potential value of this data and unable to meet the growing business needs.

[0003] In terms of user behavior data collection, the data collected by traditional methods has limited dimensions. Often, only basic operation records of users can be obtained, such as the number of clicks, the pages browsed, etc., and multi-dimensional data such as operation logs, interaction frequencies, access paths, and device status information cannot be comprehensively covered. This makes subsequent analysis lack comprehensiveness and accuracy, and it is difficult to deeply understand the true intentions and behavior patterns behind user behavior. Moreover, the control of data quality during the collection process is not strict enough, and there is a lack of effective anomaly detection and re-sampling mechanisms, resulting in the data collected may have noise, missing or errors, seriously affecting the reliability of the analysis results.

[0004] At the level of behavior feature extraction and analysis, most of the existing technologies adopt simple statistical methods or shallow machine learning algorithms, and cannot fully capture the complex temporal dependence relationships and non-linear features in user behavior data. These methods are difficult to accurately map behavior data into feature vectors, and the generated behavior feature maps cannot comprehensively and accurately reflect the behavior characteristics of users, making the identification and analysis of user behavior patterns have limitations, unable to timely discover the periodicity, relevance, and trend changes of user behavior, and it is difficult to make accurate predictions about potential risks and opportunities.

[0005] In the link of abnormal behavior detection, most systems rely on single-dimensional detection indicators or judgment methods with fixed thresholds, and cannot comprehensively consider the differences in user behavior in different dimensions and the dynamic changes of behavior patterns over time. This leads to a high false alarm rate and missed alarm rate for abnormal behavior, and it is impossible to timely and accurately identify real abnormal behaviors. In the financial field, it may lead to the inability to prevent fraud in a timely manner, and in the security field, it may not be able to effectively guarantee system security.

[0006] From the perspective of user behavior management, traditional management strategies are often static strategies formulated based on experience, lacking the ability to dynamically respond to real-time behavior data. When user behavior changes, it is impossible to timely adjust permission configurations, resource allocations, and interaction rules, and it is difficult to achieve personalized services and efficient utilization of resources. For example, in an e-commerce platform, it is not possible to dynamically adjust product recommendation strategies and inventory allocations according to users' real-time purchase behaviors and preferences, affecting user experience and the operating efficiency of enterprises.

[0007] With the continuous development of big data technology and artificial intelligence technology, new opportunities are provided to solve the above problems. However, how to organically combine these advanced technologies to build an efficient and intelligent user behavior intelligent analysis and management system has become an important issue that needs to be solved urgently. Based on such a background, the present invention aims to overcome the deficiencies of the prior art and achieve comprehensive and in-depth analysis and intelligent management of user behavior. Summary of the Invention

[0008] The purpose of the present invention is to provide a user behavior intelligent analysis and management system based on big data technology to solve the problems raised in the above background technology.

[0009] To achieve the above purpose, the present invention provides the following technical solutions: A user behavior intelligent analysis and management system based on big data technology, the system includes: A user behavior data collection module that collects user behavior data in multi-dimensional scenarios in real time, including operation logs, interaction frequencies, access paths, and device status information; A behavior feature intelligent recognition module that uses a deep neural network combined with a time series analysis algorithm to extract features from the behavior data, maps the behavior data into feature vectors, and generates a behavior feature map; A behavior pattern dynamic analysis module that divides the pattern of the feature map through a dynamic clustering algorithm and a hidden Markov model, identifies the periodicity, relevance, and trend changes of user behavior, and generates a behavior pattern dynamic analysis report; An abnormal behavior autonomous detection module that calculates the deviation degree of user behavior from the standard pattern based on a multi-dimensional abnormal scoring mechanism, and generates an abnormal behavior signal in combination with an adaptive threshold adjustment algorithm; A behavior management intelligent optimization module that optimizes the user behavior management strategy using a reinforcement learning algorithm according to the abnormal behavior signal and the behavior pattern analysis report, and dynamically adjusts the permission configuration, resource allocation, and interaction rules.

[0010] Preferably, the operation process of the user behavior data collection module includes: In the data collection stage, obtain the time series data of user operation logs through distributed sensors and log scraping interfaces, and collect the statistical values of interaction frequencies, the topological structure of access paths, and the energy consumption parameters of device status; Divide the time series data of operation logs into several data segments according to time windows, perform normalization processing on each data segment, and generate a standardized operation sequence; Perform discrete Fourier transform on the statistical values of interaction frequencies to extract frequency domain features, convert the topological structure of access paths into an adjacency matrix and calculate the node centrality index, and mark the energy consumption parameters of device status as stable state values after Gaussian filtering; If the entropy value of the standardized operation sequence, the fluctuation amplitude of the frequency domain characteristics, or the variance of the steady state value exceeds the preset acquisition threshold, a data acquisition anomaly signal is generated and the data re-acquisition process is triggered.

[0011] Preferably, the specific process of the behavioral feature intelligent recognition module includes: Construct a deep neural network model. The input layer receives the standardized operation sequence and frequency domain characteristics. The hidden layer uses a bidirectional long short-term memory network to extract temporal dependence relationships, and the output layer generates behavioral feature weights through an attention mechanism; Perform tensor fusion on the behavioral feature weights and the node centrality index of the adjacency matrix to generate a multi-dimensional feature vector; Use principal component analysis to reduce the dimension of the multi-dimensional feature vector and map it to a two-dimensional space through the t-SNE algorithm to generate a behavioral feature map; Perform density clustering on the behavioral feature map, mark the clustering centers as typical behavioral features, and mark the data points outside the clustering boundary as features to be verified.

[0012] Preferably, the specific analysis process of the behavioral pattern dynamic analysis module includes: Input the behavioral feature map into a dynamic clustering algorithm, divide the behavioral patterns based on a sliding time window, calculate the mean Euclidean distance and covariance matrix of the data points within each pattern, and generate an initial pattern set; Model the state transition probability of the pattern set through a hidden Markov model, and combine the Viterbi algorithm to predict the behavioral pattern state of the next time window; Collect the pattern similarity of adjacent time windows. If the similarity is lower than the preset pattern threshold, generate a pattern mutation signal; Compare the pattern mutation signal with the output of the trend prediction model. If the deviation exceeds the preset trend tolerance, trigger the pattern dynamic update process.

[0013] Preferably, the multi-dimensional anomaly scoring mechanism of the abnormal behavior autonomous detection module includes: Calculate the deviation degree of the user behavior feature vector from the typical behavior feature clustering center based on the Mahalanobis distance and mark it as the spatial anomaly score; Perform anomaly detection on the standardized operation sequence through the isolation forest algorithm and output the temporal anomaly score; Compare the steady state value of the device state energy consumption parameter with the historical baseline and calculate the dynamic energy consumption deviation score; Perform weighted summation on the spatial anomaly score, the temporal anomaly score, and the dynamic energy consumption deviation score to obtain a comprehensive anomaly score; If the comprehensive anomaly score exceeds the dynamic threshold output by the adaptive threshold adjustment algorithm, generate an abnormal behavior signal.

[0014] Preferably, the specific implementation of the adaptive threshold adjustment algorithm includes: Collect the mean and standard deviation of historical comprehensive anomaly scores, and calculate the initial static threshold according to the data distribution within the sliding time window; Dynamically correct the static threshold through the exponential smoothing method, and the smoothing coefficient is adaptively adjusted according to the ratio of the current data volatility to the historical volatility; If no abnormal behavior signal is detected within N consecutive time windows, the dynamic threshold is reduced at a preset decay rate; if abnormal signals are detected within M consecutive time windows, the dynamic threshold is increased at a preset increase rate.

[0015] Preferably, the reinforcement learning algorithm of the behavior management intelligent optimization module includes: Define the state space as the current user behavior pattern, abnormal signal level, and resource occupancy rate, and the action space as permission configuration adjustment, resource allocation strategy, and interaction rule parameters; Construct a Q-learning model, explore the action space through the ε-greedy strategy, and the reward function is based on the abnormal signal reduction rate, resource utilization improvement degree, and user satisfaction index; Update the Q-value table after each round of iteration. If the growth rate of the reward function in K consecutive rounds of iteration is lower than the preset optimization threshold, trigger the policy reset process.

[0016] Preferably, the calculation method of the user satisfaction index includes: Collect the feedback scores of users on permission configuration, resource response speed, and interaction rules, and normalize them to the satisfaction benchmark score; Parse the text information in the user operation log through the sentiment analysis model, extract the positive sentiment word frequency and negative sentiment word frequency, and calculate the sentiment tendency score; Perform weighted fusion on the satisfaction benchmark score and the sentiment tendency score to generate the final user satisfaction index.

[0017] Preferably, the system further includes: A behavior data archiving module, used to set the archiving period, compress and encrypt all user behavior data, feature maps, and management strategy adjustment records within the period, and store them in a distributed database; Mark duplicate data through a Bloom filter, and add noise to sensitive information using differential privacy technology to generate a secure archive file.

[0018] Preferably, the specific implementation of the differential privacy technology of the behavior data archiving module includes: Perform hash desensitization on the direct identifiers in the user behavior data, and perform k-anonymization on the indirect identifiers; Add Laplace noise to the dimensionality reduction results of the feature map, and the noise intensity is dynamically adjusted according to the data sensitivity level; Encrypt the management policy adjustment record through the homomorphic encryption algorithm, and store the key fragments in multiple secure nodes.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows: The user behavior intelligent analysis and management system based on big data technology of the present invention has many remarkable beneficial effects.

[0020] In terms of data collection, through distributed sensors and log scraping interfaces, rich behavioral data of users in multi-dimensional scenarios can be collected in real time, covering multiple dimensions such as operation logs, interaction frequencies, access paths, and device status information, comprehensively and meticulously recording user behaviors. During the collection process, the time-series data of operation logs are segmented and normalized according to time windows, and targeted processing of interaction frequencies, access paths, and device status data is carried out to ensure the standardization and effectiveness of the data. At the same time, a strict data collection anomaly detection mechanism is set up. If the entropy value of the standardized operation sequence, the fluctuation amplitude of the frequency domain characteristics, or the variance of the steady-state value exceeds the preset collection threshold, a data collection anomaly signal is generated and the data re-collection process is triggered, ensuring the high quality of the collected data and laying a solid foundation for subsequent accurate analysis.

[0021] The behavior feature intelligent recognition module constructs a deep neural network model by combining a deep neural network with a time series analysis algorithm. The deep neural network model uses a bidirectional long short-term memory network to extract time series dependencies, generates behavior feature weights through an attention mechanism, and then fuses with the node centrality index tensor of the adjacency matrix to generate a multi-dimensional feature vector. After dimensionality reduction by principal component analysis and mapping by the t-SNE algorithm, a behavior feature map is generated, and density clustering is performed to label typical behavior features and features to be verified. This method can accurately extract user behavior features, effectively capture complex behavior patterns, and make the understanding of user behaviors more in-depth and accurate.

[0022] The behavior pattern dynamic analysis module uses dynamic clustering algorithms and hidden Markov models to divide behavior patterns based on a sliding time window, calculates the mean of Euclidean distances and covariance matrices to generate an initial pattern set, and uses the hidden Markov model to model state transition probabilities and combines with the Viterbi algorithm to predict the state of behavior patterns. By collecting the pattern similarity of adjacent time windows to judge whether a pattern mutation signal is generated, and comparing it with the output of the trend prediction model, a pattern dynamic update process is triggered. This enables the system to track the changes in user behavior patterns in real time, timely discover the periodicity, correlation, and trend changes of behaviors, and provide strong support for formulating coping strategies in advance.

[0023] The abnormal behavior autonomous detection module is based on a multi-dimensional abnormal scoring mechanism. It comprehensively combines the spatial abnormal score calculated by Mahalanobis distance, the temporal abnormal score obtained by the isolation forest algorithm, and the dynamic energy consumption deviation score calculated by comparing the device status energy consumption parameters, and obtains the comprehensive abnormal score through weighted summation. Combined with the adaptive threshold adjustment algorithm, it dynamically adjusts the threshold according to the mean, standard deviation, and data volatility of the historical comprehensive abnormal score, can more accurately judge abnormal behaviors, effectively reduce the false alarm rate and missed alarm rate, and can timely detect potential risks in fields such as financial risk control and intelligent security, ensuring the safe and stable operation of the system.

[0024] The behavior management intelligent optimization module uses the reinforcement learning algorithm, defines a reasonable state space and action space, constructs a Q-learning model, explores the action space through the ε-greedy strategy, and the reward function is based on the abnormal signal reduction rate, resource utilization improvement degree, and user satisfaction index. Update the Q-value table in each iteration, and trigger the policy reset process according to the growth rate of the reward function. This enables the system to intelligently optimize the user behavior management strategy according to real-time behavior data and analysis results, dynamically adjust the permission configuration, resource allocation, and interaction rules, realize personalized services and efficient utilization of resources, and improve the user experience and enterprise operation efficiency.

[0025] In addition, the behavior data archiving module sets an archiving period, compresses and encrypts relevant data and stores it in a distributed database, uses a Bloom filter to mark duplicate data, and uses differential privacy technology to add noise to sensitive information, including desensitizing the identifiers of user behavior data, adding noise to the results of feature map dimensionality reduction, and encrypting and storing the management policy adjustment records with sharded keys, effectively ensuring the security and privacy of the data and meeting the strict requirements of data security management. Brief Description of the Drawings

[0026] Figure 1 It is the working principle diagram of the user behavior intelligent analysis and management system described in the present invention; Figure 2 It is the working flow chart of the behavior feature intelligent recognition module; Figure 3 It is the working flow chart of the behavior pattern dynamic analysis module; Figure 4 It is the schematic diagram of the scoring mechanism of the abnormal behavior autonomous detection module. Detailed Embodiments

[0027] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0028] Please refer to Figures 1-4 , the present invention provides a user behavior intelligent analysis and management system based on big data technology, and the system includes: User behavior data acquisition module: This module is responsible for real-time acquisition of user behavior data in multi-dimensional scenarios. These data include operation logs, interaction frequencies, access paths, and device status information. During actual operation, it obtains the time-series data of user operation logs through distributed sensors and log capture interfaces, and simultaneously collects the statistical values of interaction frequencies, the topological structure of access paths, and the energy consumption parameters of device status. The collected data will be further processed. For example, the time-series data of operation logs are segmented into several data segments according to time windows, and each data segment is normalized to generate a standardized operation sequence, providing a basis for subsequent analysis.

[0029] Behavior feature intelligent recognition module: This module uses a deep neural network combined with a time series analysis algorithm to extract features from the collected behavior data. It constructs a deep neural network model. The input layer receives the standardized operation sequence and frequency domain features. The hidden layer uses a bidirectional long short-term memory network to extract time series dependence relationships. The output layer generates behavior feature weights through an attention mechanism. Then, the behavior feature weights are tensor fused with the node centrality index of the adjacency matrix to generate a multi-dimensional feature vector, and a behavior feature map is generated through principal component analysis and t-SNE algorithms.

[0030] Behavior pattern dynamic analysis module: Through a dynamic clustering algorithm and a hidden Markov model, this module divides the behavior feature map into patterns. It inputs the behavior feature map into the dynamic clustering algorithm, divides the behavior patterns based on a sliding time window, calculates the mean Euclidean distance and covariance matrix of data points within each pattern, and generates an initial pattern set. Then, a hidden Markov model is used to model the state transition probability of the pattern set, and combined with the Viterbi algorithm to predict the behavior pattern state of the next time window, so as to identify the periodicity, relevance, and trend changes of user behavior, and generate a behavior pattern dynamic analysis report.

[0031] Abnormal behavior autonomous detection module: Based on a multi-dimensional abnormal scoring mechanism, this module calculates the deviation degree of user behavior from the standard pattern. It calculates the deviation degree between the user behavior feature vector and the clustering center of typical behavior features based on Mahalanobis distance, uses the isolation forest algorithm to detect anomalies in the standardized operation sequence, and calculates the dynamic energy consumption deviation score by comparing the stable state value of the device status energy consumption parameter with the historical baseline. The scores of these several dimensions are weighted and summed to obtain a comprehensive abnormal score, and combined with an adaptive threshold adjustment algorithm to generate an abnormal behavior signal.

[0032] Behavior Management Intelligent Optimization Module: Based on the abnormal behavior signals and behavior pattern analysis reports, this module uses reinforcement learning algorithms to optimize the user behavior management strategy. It defines the state space as the current user behavior pattern, abnormal signal level, and resource occupancy rate, and the action space as permission configuration adjustment, resource allocation strategy, and interaction rule parameters. It constructs a Q-learning model and guides the optimization process through a reward function to dynamically adjust the permission configuration, resource allocation, and interaction rules.

[0033] The following further details the specific working methods and collaborative processes of each module of the present invention through specific embodiments.

[0034] Embodiment 1: The operation of the user behavior data collection module is divided into multiple steps. In the data collection stage, distributed sensors and log scraping interfaces play key roles. The distributed sensors are deployed at various key nodes of the system, and they continuously monitor the user's operation behaviors, recording the obtained operation logs in the form of time series data. For example, in an online shopping system, the sensors can record the timestamps and specific operation contents of the user's operations such as clicking on product links, adding products to the shopping cart, and submitting orders. The log scraping interface extracts relevant data from the system's log files, and this data complements the data collected by the sensors to ensure that the collected operation logs are comprehensive and accurate.

[0035] At the same time, this module also collects statistical values of the interaction frequency, which can be obtained by counting the number of times the user interacts with the system within a certain time range. For example, counting the number of times the user views the product details page and chats with the customer service within one hour. For the topological structure of the access path, the system tracks the jump relationships between each page of the user to form a topological representation similar to a graph structure. Taking an e-commerce system as an example, the path from the home page to the product category page, then to the specific product details page, and finally to the settlement page constitutes an access path topological structure. And the energy consumption parameters of the device status can be obtained through the device's own energy consumption monitoring function, such as the battery consumption data of a mobile phone and the CPU power consumption data of a computer.

[0036] After the data is collected, it is necessary to process the time series data of the operation logs. It is segmented into several data segments according to time windows, and the size of the time window can be set according to actual needs, such as setting it to 5 minutes. Each data segment is normalized to eliminate the dimensional differences of the data and make the data of different data segments comparable. Assuming that the data in the operation logs includes information such as operation time, operation type, and operation object, during the normalization process, the operation time will be converted into relative time, and the operation type and operation object will be encoded, and finally a standardized operation sequence will be generated.

[0037] For the interactive frequency statistical value, the discrete Fourier transform is used to extract the frequency domain features. The formula of the discrete Fourier transform (DFT) is:

[0038] where is the time-domain signal, is the frequency-domain signal, is the length of the signal, represents the th frequency point in the frequency domain, represents the th sampling point in the time domain. Through this formula, the statistical value of the interactive frequency can be converted from the time domain to the frequency domain, and the frequency features therein can be extracted, such as whether there are periodic interactive behaviors, etc.

[0039] The topological structure of the access path is converted into an adjacency matrix, which is a matrix representing the connection relationship between nodes in a graph. If there are nodes in the graph, the element of the adjacency matrix , when there is a connection between node and node , , otherwise . Calculate the node centrality index. There are various commonly used node centrality indices, such as degree centrality, betweenness centrality, etc. Taking degree centrality as an example, for node , its degree centrality is equal to the number of nodes directly connected to this node. Through these indices, the importance of each node in the access path topological structure can be measured.

[0040] The device status energy consumption parameters are then subjected to Gaussian filtering. Gaussian filtering is a linear smoothing filter, and its purpose is to remove noise.

[0041] Finally, to ensure the accuracy and reliability of the collected data, the system performs data anomaly detection. If the entropy value of the standardized operation sequence, the fluctuation amplitude of the frequency domain features, or the variance of the steady state value exceeds the preset acquisition threshold, a data acquisition anomaly signal is generated and the data re-acquisition process is triggered. The entropy value is used to measure the uncertainty of the data. The larger the entropy value, the higher the uncertainty of the data. By setting reasonable thresholds, abnormal data can be detected in a timely manner to ensure the accuracy of subsequent analysis.

[0042] Example 2: This embodiment details the specific process of the behavioral feature intelligent recognition module. The behavioral feature intelligent recognition module first constructs a deep neural network model. The input layer of this model receives the standardized operation sequence and frequency domain features processed by the user behavior data collection module. The standardized operation sequence contains the ordered information of the user's operation behavior, and the frequency domain features reflect the frequency characteristics of the interaction frequency.

[0043] The hidden layer uses a bidirectional long short-term memory network (Bi-LSTM). Bi-LSTM can simultaneously learn the forward and backward information of the input sequence and effectively capture the temporal dependence relationship. In a text processing scenario, such as analyzing the user's input behavior in the search box, the forward information can be understood as the sequential information from the first character to the last character of the input, and the backward information is the sequential information from the last character to the first character. Through this bidirectional learning method, Bi-LSTM can more comprehensively understand the context relationship of the user's operations.

[0044] The output layer generates behavioral feature weights through the attention mechanism. The attention mechanism allows the model to automatically focus on the importance of different parts when processing data and assigns different weights to different features. For example, when analyzing the user's behavior on an e-commerce platform, for the two operations of adding a product to the shopping cart and submitting an order, the operation of submitting an order may be more important for judging the user's purchase intention, and the attention mechanism will assign higher weights to the features related to submitting an order.

[0045] Perform tensor fusion on the behavioral feature weights and the node centrality index of the adjacency matrix. Tensor fusion is to merge two or more tensors into a new tensor according to certain rules. The node centrality index of the adjacency matrix reflects the importance of each node in the access path. After fusing with the behavioral feature weights, it can comprehensively consider the user's operation behavior and the structural information of the access path to generate a more representative multi-dimensional feature vector.

[0046] After obtaining the multi-dimensional feature vector, principal component analysis (PCA) is used for dimensionality reduction. The purpose of PCA is to reduce the dimension of the data and the computational complexity while trying to retain the data information as much as possible. The main steps of PCA include calculating the covariance matrix of the data, solving the eigenvalues and eigenvectors of the covariance matrix, and then selecting the eigenvectors corresponding to the first largest eigenvalues to form a projection matrix, and projecting the original data into this low-dimensional space. Suppose the original data matrix is , the data matrix after PCA dimensionality reduction is , and the projection matrix is . Then there is

[0047] The feature vectors after dimensionality reduction are mapped to a two-dimensional space through the t-SNE algorithm to generate a behavioral feature map. The t-SNE (t-Distributed Stochastic Neighbor Embedding) algorithm is a dimensionality reduction technique for data visualization that can map high-dimensional data into a two-dimensional or three-dimensional space while trying to maintain the relative distance relationship between the data. In the generated behavioral feature map, similar behavioral features will cluster together, while different behavioral features will be separated, facilitating subsequent analysis and understanding.

[0048] Density clustering is performed on the behavioral feature map. Density clustering is a method of dividing clusters based on the density distribution of data points. In the behavioral feature map, regions with higher density are divided into one cluster, and the cluster center is marked as a typical behavioral feature, which represents the common and representative behavioral patterns of users. Data points outside the cluster boundary are marked as features to be verified, and these features may be manifestations of abnormal behaviors and require further analysis and verification.

[0049] Example 3: This example details the specific analysis process of the behavioral pattern dynamic analysis module. The behavioral pattern dynamic analysis module first inputs the behavioral feature map into the dynamic clustering algorithm. The dynamic clustering algorithm divides behavioral patterns based on a sliding time window, and the setting of the sliding time window is to adapt to the changes in behavioral patterns over time. For example, when analyzing a user's behavior on a social media platform, a 1-hour sliding time window is used, and it slides every 15 minutes. Within each time window, the mean Euclidean distance and covariance matrix of the data points within each pattern are calculated. The Euclidean distance is used to measure the distance between two data points in space. By calculating the mean Euclidean distance, the degree of aggregation of the data points within the pattern can be understood; the covariance matrix reflects the correlation between the data points. Based on these calculation results, an initial pattern set is generated.

[0050] Then, the state transition probability is modeled for the pattern set through the Hidden Markov Model (HMM). The HMM is a statistical model that assumes the system is a Markov process, that is, the next state of the system depends only on the current state and is independent of the previous states. In this system, each behavioral pattern can be regarded as a state in the HMM. The HMM has three important parameters: the state transition probability matrix , which represents the probability of transitioning from one state to another; the observation probability matrix , which represents the probability of generating a specific observation value in a certain state; the initial state probability vector , which represents the probability that the system is in each state at the initial moment. By learning the pattern set, the values of these parameters can be determined.

[0051] Predict the behavior pattern state of the next time window in combination with the Viterbi algorithm. The Viterbi algorithm is an algorithm for finding the most likely state sequence in an HMM. It uses dynamic programming to calculate the maximum probability path to each state at each time step and finally finds the state sequence with the highest probability. In this system, the Viterbi algorithm can be used to predict the most likely behavior pattern of the user in the next time window based on the current behavior pattern state and the state transition probability.

[0052] During the analysis process, collect the pattern similarity of adjacent time windows. The pattern similarity can be measured by calculating the overlap degree of the behavior patterns or the similarity of the feature vectors in two time windows. For example, the cosine similarity is used to calculate the similarity of the feature vectors. If the similarity is lower than the preset pattern threshold, a pattern mutation signal is generated. This means that the user's behavior pattern has changed significantly, and there may be a new behavior trend or abnormal behavior.

[0053] Finally, compare the pattern mutation signal with the output of the trend prediction model. The trend prediction model can be a model constructed based on time series analysis or machine learning algorithms, which is used to predict the long-term trend of the behavior pattern. If the deviation exceeds the preset trend tolerance, the pattern dynamic update process is triggered. This is to timely adjust the division and analysis model of the behavior pattern to adapt to the changing user behavior.

[0054] Example 4: The multi-dimensional anomaly scoring mechanism of the abnormal behavior autonomous detection module evaluates the deviation degree of the user behavior from the standard pattern from multiple perspectives. First, calculate the deviation degree between the user behavior feature vector and the clustering center of the typical behavior features based on the Mahalanobis distance, and mark it as the spatial anomaly score. The Mahalanobis distance takes into account the covariance structure of the data and can more accurately measure the similarity between data points. If the Mahalanobis distance between the user's behavior feature vector and the clustering center of the typical behavior features is large, it means that the behavior deviates from the normal pattern in space, and the spatial anomaly score will be high.

[0055] Perform anomaly detection on the standardized operation sequence through the Isolation Forest algorithm and output the temporal anomaly score. The Isolation Forest algorithm is an anomaly detection algorithm based on the idea of isolation. It constructs a binary tree to isolate data points. For a data point, if its path length in the binary tree is short, it means that it is easily isolated and is more likely to be an anomaly point. The Isolation Forest algorithm calculates an anomaly score for each data point, and this score is the temporal anomaly score.

[0056] Compare the steady-state value of the device status energy consumption parameter with the historical baseline to calculate the dynamic energy consumption deviation score. The historical baseline can be the average value of the device status energy consumption parameter over a past period of time or a curve fitted based on historical data. The dynamic energy consumption deviation score can be obtained by calculating the difference or relative difference between the current steady-state value and the historical baseline. For example, if the current steady-state value is , and the historical baseline value is , the dynamic energy consumption deviation score can be defined as .

[0057] Weightedly sum the spatial anomaly score, the temporal anomaly score, and the dynamic energy consumption deviation score to obtain the comprehensive anomaly score. Suppose the spatial anomaly score is , the temporal anomaly score is , the dynamic energy consumption deviation score is , and the corresponding weights are , , , then the comprehensive anomaly score . These weights can be determined based on the importance of different dimensions through experience or machine learning algorithms.

[0058] If the comprehensive anomaly score exceeds the dynamic threshold output by the adaptive threshold adjustment algorithm, an abnormal behavior signal is generated. The specific implementation of the adaptive threshold adjustment algorithm is as follows: First, collect the mean and standard deviation of the historical comprehensive anomaly scores, and calculate the initial static threshold according to the data distribution within the sliding time window. Suppose the historical comprehensive anomaly score data is , the mean is , the standard deviation is , is the total number of historical comprehensive anomaly score data. The initial static threshold can be set according to the mean and standard deviation. For example, it is set to , is an empirical value.

[0059] Then, dynamically correct the static threshold through the exponential smoothing method, and the smoothing coefficient is adaptively adjusted according to the ratio of the current data volatility to the historical volatility. The formula of the exponential smoothing method is:

[0060] where is the actual value at the current moment, is the predicted value at the current moment (i.e., the corrected threshold), is the predicted value at the previous moment, is the smoothing coefficient. The current data volatility and the historical volatility can be measured by calculating the change rate of the variance or standard deviation of the data. If the current data volatility is large, it means that the data changes relatively violently, and the smoothing coefficient It will increase correspondingly, making the threshold more adaptable to the changes in the data.

[0061] If no abnormal behavior signal is detected within consecutive time windows, the dynamic threshold is decreased at a preset attenuation rate; if abnormal signals are detected within consecutive time windows, the dynamic threshold is increased at a preset amplification rate. In this way, the size of the threshold can be dynamically adjusted according to the actual occurrence of abnormal behaviors in the system, improving the accuracy and adaptability of abnormal detection.

[0062] Embodiment 5: This embodiment details the reinforcement learning algorithm of the behavior management intelligent optimization module and the calculation method of the user satisfaction index.

[0063] The behavior management intelligent optimization module uses a reinforcement learning algorithm to optimize the user behavior management strategy. First, the state space is defined. The state space includes the current user behavior pattern, the abnormal signal level, and the resource occupancy rate. The current user behavior pattern can be obtained through the behavior pattern dynamic analysis module, which reflects the current behavior characteristics of the user; the abnormal signal level is determined according to the output of the abnormal behavior self-detection module and is used to represent the severity of the abnormal behavior; the resource occupancy rate represents the proportion of system resources (such as CPU usage rate, memory occupancy rate, etc.) occupied.

[0064] The action space is defined as permission configuration adjustment, resource allocation strategy, and interaction rule parameters. Permission configuration adjustment can include increasing or decreasing the user's access permissions to certain functions. For example, in an enterprise management system, the read and write permissions of employees to specific files are adjusted; the resource allocation strategy involves how to allocate the system's computing resources, storage resources, etc., such as allocating different proportions of server memory to different business modules; the interaction rule parameters are used to set the rules for user-system interaction, such as adjusting the response time of interface elements and limiting the operation frequency.

[0065] A Q-learning model is constructed, which is a reinforcement learning algorithm based on the value function. The action space is explored through the ε-greedy strategy. The ε-greedy strategy randomly selects an action with a probability of ε and selects the action with the maximum current Q value with a probability of 1 - ε. This can ensure that the algorithm has a certain degree of exploration to try new actions and can also use existing experience to select the current optimal action.

[0066] The reward function is based on the abnormal signal reduction rate, the resource utilization improvement degree, and the user satisfaction index. The abnormal signal reduction rate is used to measure the reduction of abnormal behavior signals in the system after taking a certain action. The calculation formula is: Abnormal signal reduction rate = (Number of abnormal signals before taking the action - Number of abnormal signals after taking the action) ÷ Number of abnormal signals before taking the action. The resource utilization improvement degree reflects the impact of the resource allocation strategy on the resource utilization efficiency of the system. For example, Resource utilization improvement degree = (Resource utilization rate after taking the action - Resource utilization rate before taking the action) ÷ Resource utilization rate before taking the action.

[0067] The calculation method of the user satisfaction index is as follows: Collect the feedback scores of users on permission configuration, resource response speed, and interaction rules, and normalize them to the satisfaction benchmark score. Suppose the score of the user on permission configuration is and the score on resource response speed is and the score on interaction rules is , and perform normalization through the formula: . The full score is determined according to the score range. For example, if the score is from 1 to 5, the full score is 5.

[0068] Parse the text information in the user operation log through the sentiment analysis model, extract the positive sentiment word frequency and negative sentiment word frequency, and calculate the sentiment tendency score. The sentiment analysis model can be based on natural language processing technology, such as using the bag-of-words model combined with machine learning algorithms or deep learning models (such as convolutional neural networks, recurrent neural networks, etc.) to judge the sentiment tendency in the text. Suppose the positive sentiment word frequency is and the negative sentiment word frequency is , and the sentiment tendency score can be calculated through the formula: . When the positive sentiment word frequency is greater than the negative sentiment word frequency, the sentiment tendency score is positive, indicating that the user has a positive sentiment; otherwise, it is negative.

[0069] Weightedly fuse the satisfaction benchmark score and the sentiment tendency score to generate the final user satisfaction index. Let the weight of the satisfaction benchmark score be and the weight of the sentiment tendency score be , and the final user satisfaction index = Satisfaction benchmark score + Sentiment tendency score. The weights and can be determined according to the actual situation through experience or machine learning algorithms to balance the influence of direct user feedback and text sentiment analysis results on the satisfaction index.

[0070] After each iteration, the Q-value table is updated. The Q-value table records the value estimates for taking each action in each state. If the growth rate of the reward function is lower than the preset optimization threshold for three consecutive iterations, the policy reset process is triggered to re-explore a better policy to adapt to the changes in the system environment and the dynamic characteristics of user behavior.

[0071] Example 6: The behavior data archiving module first sets the archiving period, which can be flexibly set according to actual needs, such as daily, weekly, or monthly. During each archiving period, the module collects and organizes all user behavior data, feature maps, and management policy adjustment records within the period. Taking an online education platform as an example, user behavior data includes students' course learning duration, homework submission status, interaction records with teachers, etc.; the feature map is a map reflecting students' learning behavior characteristics generated by the behavior feature intelligent recognition module; the management policy adjustment records cover adjustments to students' permissions (such as whether to grant advanced course permissions), resource allocation (such as allocating different server bandwidths for different students for video learning), and interaction rules (such as adjusting the operation response time of the learning interface), etc.

[0072] After collecting the data, these data are compressed and encrypted, and then stored in a distributed database. The purpose of compression is to reduce the space required for data storage and improve storage efficiency. Common compression algorithms such as ZIP, GZIP, etc. can be used. Encryption is to ensure data security and prevent data leakage. Encryption algorithms such as AES (Advanced Encryption Standard) with high security performance can be selected.

[0073] During the data processing, duplicate data is marked through a Bloom filter. A Bloom filter is a probabilistic data structure that can efficiently determine whether an element is in a set. Its principle is to map an element to a bit array through multiple hash functions and set the corresponding bits to 1. When determining whether an element is in the set, calculate its position in the bit array through the same hash function. If all corresponding positions are 1, it is considered that the element may be in the set (there is a certain false positive rate, but the false positive rate can be controlled by adjusting parameters); if any one position is 0, the element must not be in the set. In behavior data archiving, the Bloom filter can quickly mark duplicate data, avoid duplicate storage, and save storage space.

[0074] In addition, differential privacy technology is used to add noise to sensitive information to generate a secure archived file. The differential privacy technology is mainly implemented from the following aspects: Hash desensitization is performed on the direct identifiers in the user behavior data. Direct identifiers such as the user's ID card number, mobile phone number, etc. Hash desensitization is to convert these sensitive information through a hash function to obtain an irreversible hash value, thereby protecting the user's privacy. K-anonymization is performed on the indirect identifiers. Indirect identifiers such as the combined information of the user's age, gender, occupation, etc. K-anonymization is to make at least k records in the dataset have the same combination of indirect identifiers through generalization or concealment, so that it is difficult for attackers to determine the specific user identity through the indirect identifiers.

[0075] Laplace noise is added to the dimensionality reduction result of the feature map, and the noise intensity is dynamically adjusted according to the data sensitivity level. By adding Laplace noise, even if an attacker obtains the feature map, it is difficult to accurately infer the user's true behavior characteristics from the map.

[0076] For the management policy adjustment records, encryption is performed through the homomorphic encryption algorithm. Homomorphic encryption allows specific calculations to be performed on ciphertext, and the result after decryption is the same as the result of performing the same calculation on plaintext. This means that in the case of encrypted data storage, the system can still perform necessary analysis and processing on the management policy adjustment records without decrypting the data, further ensuring the security of the data. The key is shard-stored in multiple secure nodes, so that even if the key of a certain node is leaked, it will not cause the entire encryption system to fail, improving the security of the key and the reliability of the system.

[0077] All the data involved in this application are obtained with full consent and authorization, and the collection, use, and processing of relevant information need to comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0078] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0079] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A user behavior intelligent analysis and management system based on big data technology, characterized in that, include: User behavior data collection module, which collects user behavior data in multi-dimensional scenarios in real time, including operation logs, interaction frequency, access path and device status information; The behavior feature intelligent recognition module uses a deep neural network combined with a time series analysis algorithm to extract features from behavior data, map the behavior data into feature vectors, and generate a behavior feature map; The behavior pattern dynamic analysis module uses dynamic clustering algorithms and hidden Markov models to divide the feature graph into patterns, identify the periodicity, correlation and trend changes of user behaviors, and generate a behavior pattern dynamic analysis report; The abnormal behavior autonomous detection module calculates the deviation of user behavior from the standard mode based on the multi-dimensional abnormal scoring mechanism, and generates abnormal behavior signals in combination with the adaptive threshold adjustment algorithm; The behavior management intelligent optimization module uses reinforcement learning algorithms to optimize user behavior management strategies based on abnormal behavior signals and behavior pattern analysis reports, and dynamically adjusts permission configuration, resource allocation, and interaction rules.

2. The user behavior intelligent analysis and management system based on big data technology according to claim 1, wherein The operation process of the user behavior data collection module includes: In the data collection phase, the distributed sensors and log capture interface are used to obtain the time series data of user operation logs, collect the statistical values of interaction frequency, the topological structure of access paths, and the energy consumption parameters of device status; The time series data of the operation log is divided into several data segments according to the time window, and each data segment is normalized to generate a standardized operation sequence; Perform discrete Fourier transform on the interaction frequency statistics to extract frequency domain features, convert the topological structure of the access path into an adjacency matrix and calculate the node centrality index, and mark the device state energy consumption parameters as stable state values after Gaussian filtering; If the entropy value of the standardized operation sequence, the fluctuation amplitude of the frequency domain characteristics, or the variance of the steady state value exceeds the preset acquisition threshold, a data acquisition abnormality signal is generated and the data re-acquisition process is triggered.

3. The user behavior intelligent analysis and management system based on big data technology according to claim 1, wherein The specific process of the behavior feature intelligent recognition module includes: Construct a deep neural network model, where the input layer receives standardized operation sequences and frequency domain features, the hidden layer uses a bidirectional long short-term memory network to extract temporal dependencies, and the output layer generates behavioral feature weights through an attention mechanism; The behavior feature weights are tensor-fused with the node centrality index of the adjacency matrix to generate a multi-dimensional feature vector; The multi-dimensional feature vector is reduced in dimension using principal component analysis, and then mapped to two-dimensional space using the t-SNE algorithm to generate a behavioral feature map. The behavioral feature map is density clustered, the cluster center is marked as a typical behavioral feature, and the data points outside the cluster boundary are marked as features to be verified.

4. The user behavior intelligent analysis and management system based on big data technology according to claim 1, characterized in that, The specific analysis process of the behavior pattern dynamic analysis module includes: The behavior feature map is input into the dynamic clustering algorithm, the behavior patterns are divided based on the sliding time window, the mean Euclidean distance and covariance matrix of the data points in each pattern are calculated, and the initial pattern set is generated; The state transition probability of the pattern set is modeled by the hidden Markov model, and the behavior pattern state of the next time window is predicted by combining the Viterbi algorithm; Collect the pattern similarity of adjacent time windows, and if the similarity is lower than the preset pattern threshold, generate a pattern mutation signal; Compare the pattern mutation signal with the output of the trend prediction model. If the deviation exceeds the preset trend tolerance, trigger the pattern dynamic update process.

5. The user behavior intelligent analysis and management system based on big data technology according to claim 1, characterized in that The multi-dimensional anomaly scoring mechanism of the abnormal behavior autonomous detection module includes: Calculate the deviation degree between the user behavior feature vector and the typical behavior feature clustering center based on the Mahalanobis distance, and mark it as the spatial anomaly score; Perform anomaly detection on the standardized operation sequence through the isolation forest algorithm, and output the time series anomaly score; Compare the stable state value of the device status energy consumption parameter with the historical baseline, and calculate the dynamic energy consumption deviation score; Perform weighted summation on the spatial anomaly score, the time series anomaly score, and the dynamic energy consumption deviation score to obtain the comprehensive anomaly score; If the comprehensive anomaly score exceeds the dynamic threshold output by the adaptive threshold adjustment algorithm, generate an abnormal behavior signal.

6. The user behavior intelligent analysis and management system based on big data technology according to claim 5, wherein The specific implementation of the adaptive threshold adjustment algorithm includes: Collect the mean and standard deviation of the historical comprehensive anomaly scores, and calculate the initial static threshold according to the data distribution within the sliding time window; Dynamically correct the static threshold through the exponential smoothing method, and the smoothing coefficient is adaptively adjusted according to the ratio of the current data volatility and the historical volatility; If no abnormal behavior signal is detected within N consecutive time windows, reduce the dynamic threshold according to the preset decay rate; if abnormal signals are detected within M consecutive time windows, increase the dynamic threshold according to the preset increase rate.

7. The user behavior intelligent analysis and management system based on big data technology according to claim 1, characterized in that The reinforcement learning algorithm of the behavior management intelligent optimization module includes: Define the state space as the current user behavior pattern, the abnormal signal level, and the resource occupancy rate, and the action space as the permission configuration adjustment, the resource allocation strategy, and the interaction rule parameters; Construct a Q-learning model, explore the action space through the ε-greedy strategy, and the reward function is based on the abnormal signal reduction rate, the resource utilization improvement degree, and the user satisfaction index; Update the Q-value table after each round of iteration. If the growth rate of the reward function in K consecutive rounds of iteration is lower than the preset optimization threshold, trigger the policy reset process.

8. The user behavior intelligent analysis and management system based on big data technology according to claim 7, characterized in that, The calculation method of the user satisfaction index includes: Collect the feedback scores of the user on the permission configuration, the resource response speed, and the interaction rules, and normalize them into the satisfaction benchmark score; Parse the text information in the user operation log through the sentiment analysis model, extract the positive sentiment word frequency and the negative sentiment word frequency, and calculate the sentiment tendency score; Perform weighted fusion on the satisfaction benchmark score and the sentiment tendency score to generate the final user satisfaction index.

9. The user behavior intelligent analysis and management system based on big data technology according to claim 1, characterized in that It also includes: The behavior data archiving module is used to set the archiving period, compress and encrypt all user behavior data, feature maps, and management policy adjustment records within the period, and store them in the distributed database; Mark duplicate data through the Bloom filter, and add noise to sensitive information using differential privacy technology to generate a secure archive file.

10. The user behavior intelligent analysis and management system based on big data technology according to claim 9, characterized in that The specific implementation of the differential privacy technology of the behavior data archiving module includes: Perform hash desensitization on the direct identifiers in the user behavior data, and perform k-anonymization processing on the indirect identifiers; Add Laplace noise to the dimensionality reduction result of the feature map, and the noise intensity is dynamically adjusted according to the data sensitivity level; Encrypt the management policy adjustment records through the homomorphic encryption algorithm, and the key is sharded and stored in multiple secure nodes.

Images