A user portrait generation method and system based on big data

Abstract

The application relates to the technical field of big data analysis and user behavior modeling, and provides a user portrait generation method and system based on big data. The method comprises the following steps: extracting time series and interaction frequency features from user behavior data, constructing a user access mode feature set and analyzing data missing distribution; extracting boundary features based on the missing distribution; if the boundary features are significantly associated with complete data, surrounding data is detected to identify behavior rules; according to the behavior rules, a potential associated path in a missing area is extracted, and a user behavior mode prototype is inferred; the behavior rules are propagated to the missing area to generate data inference values; path and boundary features are fused to determine the internal structure of the missing area, and a user activity sequence is generated; and the internal structure and the inference values are verified for consistency to form a complete user feature model. The application infers missing data through context, and improves the accuracy and completeness of the user portrait.

Description

Technical Field

[0001] This invention relates to the field of big data analysis and user behavior modeling technology, and in particular to a method and system for generating user profiles based on big data. Background Technology

[0002] User profiling is a key technology in the field of big data. By analyzing user behavior and characteristics, it provides crucial support for scenarios such as personalized recommendations and precision marketing. This technology plays an irreplaceable role in improving enterprise decision-making efficiency and optimizing user experience. With the rapid growth of data scale and the increasing complexity of application scenarios, user profiling requires extracting meaningful behavioral patterns from massive, multi-dimensional data.

[0003] In existing technologies, user profiling typically relies on statistical analysis of user behavior logs, clustering algorithms, or machine learning models. For example, by extracting features such as user click sequences, dwell time, and access paths, interest inference can be performed using collaborative filtering or deep learning models.

[0004] However, existing technologies often employ simple imputation strategies when dealing with missing data, such as mean imputation, nearest neighbor imputation, or rule-based inference methods. While these methods can complete the data to some extent, they fail to fully consider the structural characteristics of the missing data and its correlation with user behavior patterns. This leads to imputation results that deviate from actual user behavior, thus affecting the accuracy and completeness of the user profile. In summary, existing technologies lack effective mining of the boundary features and contextual relationships of missing data regions, causing inference results to deviate from actual user behavior and limiting the accuracy and reliability of user profile generation. Summary of the Invention

[0005] This invention provides a user profile generation method and system based on big data, so as to effectively mine the boundary features and contextual relationships of missing data regions.

[0006] Firstly, in order to solve the above-mentioned technical problems, the present invention provides a user profile generation method based on big data, comprising:

[0007] Obtain raw user behavior data, extract time series features and interaction frequency features from the raw user behavior data and analyze them to determine the data missing distribution;

[0008] Based on the missing data distribution, boundary analysis is performed on the surrounding complete data, and boundary features are extracted.

[0009] If the correlation between the boundary feature and the surrounding complete data is higher than the preset correlation strength significance threshold, then the user behavior distribution pattern is obtained by detecting the surrounding data of the abnormal pattern group.

[0010] According to the user behavior distribution law, the potential correlation path of the data missing area is extracted, and a user behavior mode sketch is obtained by judging;

[0011] Based on the user behavior mode sketch, the user behavior distribution law is propagated to the data missing area to obtain a missing data inference value;

[0012] According to the missing data inference value, the potential correlation path is integrated, and if the missing data inference value matches the boundary feature, the surrounding complete data is fused to determine the internal structure of the data missing area;

[0013] According to the internal structure of the data missing area and the missing data inference value, a user activity sequence is generated by analyzing the correlation weight;

[0014] According to the user activity sequence, the consistency of the internal structure of the data missing area and the missing data inference value is verified, and a complete user feature model is obtained.

[0015] Preferably, the user behavior original data is obtained, the time sequence feature and the interaction frequency feature are extracted from the user behavior original data and analyzed, and the data missing distribution is determined, including:

[0016] The user behavior original data is obtained from the user behavior log database, the behavior sequence is generated by timestamp sorting, and the user behavior time sequence data set is obtained;

[0017] According to the user behavior time sequence data set, the time interval of adjacent access time is calculated, and the click number and interaction frequency are counted to obtain an interaction frequency feature set;

[0018] If the click number in the interaction frequency feature set is lower than the preset device interaction effectiveness threshold, the access path and session duration data are extracted and grouped to obtain a user access mode feature set;

[0019] For the user access mode feature set, the missing proportion of time interval and operation type is counted, and it is judged whether the missing data is concentrated in a specific device type or time interval to obtain a data missing distribution feature set.

[0020] Preferably, according to the data missing distribution, the boundary of the surrounding complete data is analyzed, and the boundary feature is extracted, including:

[0021] Based on the data missing distribution feature set, user location trajectory data and event correlation matrix data are obtained, and complete data boundaries are identified to obtain a time connection point set;

[0022] According to the time connection point set, the distribution frequency of the user activity event on the location point is counted to obtain a location connection point set;

[0023] For the location connection point set and the time connection point set, a time-location correlation graph of the user activity is constructed, and nodes in the graph are grouped to obtain a boundary feature set;

[0024] From the boundary feature set, the proportion of the time connection point and the location connection point in the data missing distribution feature set is counted, and if the proportion is lower than a preset data missing distribution proportion threshold, a supplementary feature is extracted from complete data to obtain a boundary feature of the user activity.

[0025] Preferably, if the association degree of the boundary feature and the surrounding complete data is higher than a preset association strength significance threshold, surrounding data is grouped through abnormal mode detection to obtain a user behavior distribution rule, including:

[0026] Based on the boundary feature, the association strength of the time point and the location point in the ordered data set is calculated to obtain an association strength set;

[0027] If the association strength of a certain time point and a location point in the association strength set is higher than a preset association strength significance threshold, the corresponding user activity event is extracted and grouped to obtain an abnormal event set;

[0028] According to the abnormal event set, the frequency distribution of the user activity event in the time sequence and the location trajectory is counted and grouped to obtain a user behavior distribution rule set;

[0029] For the user behavior distribution rule set, the distribution proportion of the distribution rule in the time sequence and the location trajectory is calculated, and if the distribution proportion is lower than a preset distribution proportion threshold, a supplementary event feature is extracted from complete data to obtain a final user behavior distribution rule.

[0030] Preferably, according to the user behavior distribution rule, a potential association path of the data missing area is extracted and judged to obtain a user behavior mode sketch, including:

[0031] Based on the user behavior distribution rule, if the association strength of a certain time point and a location point in the association strength set is lower than a preset association strength significance threshold, a data missing area is extracted, and a potential path set is identified and obtained;

[0032] According to the potential path set, the event frequency distribution is calculated and grouped to obtain a user activity classification set;

[0033] For the user activity classification set, a behavior mode framework is constructed in combination with the time sequence feature and the location trajectory association to judge the user mode sketch.

[0034] Preferably, based on the user behavior pattern sketch, the user behavior distribution law is propagated to a data missing area to obtain a missing data inference value, including:

[0035] Based on the user behavior pattern sketch as prior knowledge, a statistical pattern in the user behavior distribution law set is taken as label information and is propagated to an unlabeled node corresponding to the data missing area through an edge in the time-location correlation graph.

[0036] According to a distribution probability converged on the unlabeled node, an active event corresponding to a maximum probability is taken as a missing data inference value of the data missing area.

[0037] Preferably, the potential correlation path is integrated according to the missing data inference value, and if the missing data inference value matches the boundary feature, the surrounding complete data is fused to determine an internal structure of the data missing area, including:

[0038] The missing data inference value is filled into the potential path set to generate a completed candidate path set.

[0039] A matching degree of the candidate path set and the boundary feature set is calculated, and if the matching degree is higher than a preset matching threshold, it is determined that the matching path boundary consistency is consistent.

[0040] Based on the matching path boundary consistency, the correlation features of the surrounding complete data are fused, a weighted value of each feature point in the data missing area is calculated by taking the matching degree as a weight, and the internal structure of the data missing area is determined.

[0041] Preferably, a user activity sequence is generated by correlation weight analysis according to the internal structure of the data missing area and the missing data inference value, including:

[0042] Based on the internal structure of the missing area and the missing data inference value, a time sequence reorganization data set sorted and spliced according to a timestamp is obtained.

[0043] Based on the time sequence reorganization data set, a transfer law of an active event in time is analyzed, a most possible user activity sequence is decoded, and an initial user activity sequence is obtained.

[0044] The initial user activity sequence is subjected to smoothing filtering processing to generate a final user activity sequence.

[0045] Preferably, the consistency of the internal structure of the data missing area and the missing data inference value is verified according to the user activity sequence to obtain a complete user feature model, including:

[0046] extracting a behavior segment of a region filled by the missing data inferred value as a to-be-verified sequence segment based on the user activity sequence;

[0047] calculating a behavior feature matching degree between the to-be-verified sequence segment and the missing data inferred value, and checking logical coherence with the boundary feature;

[0048] if the behavior feature matching degree is higher than a preset consistency threshold and the logical coherence checking passes, determining that the internal structure of the data missing region is consistent with the missing data inferred value, and obtaining a consistency determination result;

[0049] based on the consistency determination result, fusing the user activity sequence and an interest label to generate a complete user feature model.

[0050] In a second aspect, the present application provides a user portrait generation system based on big data, comprising:

[0051] a data acquisition and preprocessing module, which extracts time sequence features and interaction frequency features from user behavior raw data to obtain a user access mode feature set, and further determines a data missing distribution feature set;

[0052] a boundary feature extraction module, which processes surrounding complete data and extracts boundary features according to the data missing distribution feature set;

[0053] a behavior rule discovery module, which detects surrounding data to obtain a user behavior distribution rule if the association degree of the boundary feature with complete data is higher than an association strength significance threshold;

[0054] a mode sketch identification module, which extracts a potential association path of a data missing region according to the user behavior distribution rule to determine a user behavior mode sketch;

[0055] a missing data inference module, which propagates the user behavior distribution rule to a data missing region based on the user behavior mode sketch to obtain a missing data inferred value;

[0056] an internal structure construction module, which integrates the potential association path according to the missing data inferred value, and if the potential association path matches the boundary feature, fuses surrounding data association to determine the internal structure of the data missing region;

[0057] a user activity sequence generation module, which analyzes and generates a user activity sequence according to the internal structure of the data missing region and the missing data inferred value;

[0058] a consistency checking and model generation module, which verifies the consistency of the internal structure of the data missing region and the missing data inferred value according to the user activity sequence, and if consistent, obtains a complete user feature model.

[0059] In a third aspect, the present application also provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the user portrait generation method based on big data according to any one of the above embodiments when executing the computer program.

[0060] In a fourth aspect, the present application also provides a computer readable storage medium comprising a stored computer program, wherein the computer readable storage medium controls the device where the computer readable storage medium is located to execute the user portrait generation method based on big data according to any one of the above embodiments when the computer program runs.

[0061] Compared with the prior art, the present application has the following beneficial effects:

[0062] (1) The present application integrates the user behavior context in the inference of missing data by identifying boundary features, constructing a time-location association graph, and judging whether to perform regular propagation based on an association strength threshold, thereby effectively avoiding the noise introduced by simple filling and improving the accuracy of missing data inference and its consistency with the user's real behavior.

[0063] (2) The present application ensures the connection of the missing area with the existing behavior logic through potential path identification and behavior pattern propagation, and verifies the overall consistency through sequence continuity representation, forming a coherent and complete user behavior sequence, which significantly improves the integrity and reliability of the user portrait and is suitable for advanced application scenarios such as personalized recommendation.

[0064] (3) The present application systematically handles different types and levels of missing problems through hierarchical judgment, threshold control such as association significance threshold, distribution proportion threshold, and structured propagation mechanism, improves the adaptability of the system to complex data missing scenarios, and reduces redundant calculation through process optimization to improve the processing efficiency and system stability in the big data environment. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 is a user portrait generation method based on big data provided by the first embodiment of the present application;

[0066] Figure 2 is a user portrait generation system structure diagram based on big data provided by the second embodiment of the present application. DETAILED DESCRIPTION

[0067] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described, obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0068] With reference to Figure 1 The first embodiment of the present application provides a user portrait generation method based on big data, comprising the following steps:

[0069] S11, acquiring user behavior original data, extracting time sequence features and interaction frequency features from the user behavior original data and analyzing to determine data missing distribution;

[0070] S12, according to the data missing distribution, performing boundary analysis on surrounding complete data, and extracting boundary features;

[0071] S13, if the correlation degree of the boundary features and the surrounding complete data is higher than a preset correlation strength significance threshold, grouping surrounding data through abnormal mode detection to obtain user behavior distribution rule;

[0072] S14, according to the user behavior distribution rule, extracting potential correlation paths of data missing areas and judging to obtain user behavior mode sketch;

[0073] S15, based on the user behavior mode sketch, propagating the user behavior distribution rule to the data missing area to obtain missing data inference value;

[0074] S16, integrating the potential correlation paths according to the missing data inference value, if the missing data inference value matches the boundary features, fusing the surrounding complete data to determine the internal structure of the data missing area;

[0075] S17, according to the internal structure of the data missing area and the missing data inference value, generating user activity sequence through correlation weight analysis;

[0076] S18, according to the user activity sequence, verifying the consistency of the internal structure of the data missing area and the missing data inference value to obtain a complete user feature model.

[0077] In step S11, the user behavior original data is acquired, time sequence features and interaction frequency features are extracted from the user behavior original data and analyzed to determine data missing distribution, comprising:

[0078] S1101, obtain user behavior original data, generate a behavior sequence by timestamp sorting to obtain a user behavior time series dataset;

[0079] S1102, calculate a time interval of adjacent access times and count a click number and an interaction frequency according to the user behavior time series dataset to obtain an interaction frequency feature set;

[0080] S1103, if the click number in the interaction frequency feature set is lower than a preset device interaction effectiveness threshold, extract access path and session duration data and group them to obtain a user access mode feature set;

[0081] S1104, for the user access mode feature set, count a missing proportion of a time interval and an operation type and judge whether missing data is concentrated in a specific device type or time interval to obtain a data missing distribution feature set.

[0082] In step S1101, user behavior original data is obtained, a behavior sequence is generated by timestamp sorting, and a user behavior time series dataset is obtained.

[0083] It should be noted that the user original data is obtained from a user behavior log database, including user identification, access time, operation type, page stay, access path, device type, and click number.

[0084] In an embodiment, in a user behavior analysis scenario of an information service platform, key data needs to be extracted from a user behavior log database and a behavior sequence needs to be generated. The original data includes user identification, access time, operation type, page stay duration, access path, device type, and click number. These data can be sorted by timestamp to generate a user behavior time series dataset. For example, user A browses the home page at 2025-09-08 10:00:00, clicks the information detail page at 10:01:30, and collects the information at 10:02:00. The behavior sequence is "browse home page→click information detail→collect information", and the timestamp sorting ensures that the behaviors are accurately arranged in chronological order. The significance of this step is to reflect the continuity of user behavior through the time series, which is convenient for subsequent analysis of user habits and patterns.

[0085] In step S1102, according to the user behavior time series dataset, the time interval of adjacent access times is calculated, and the click number and the interaction frequency are counted to obtain the interaction frequency feature set.

[0086] It should be noted that according to the user behavior time series data set, when calculating the adjacent access time interval, the timestamp of each operation can be extracted from the time series, and the time difference between two operations is calculated. If the page dwell time exceeds the page dwell time threshold, it is marked as an abnormal dwell, and the interaction frequency feature set is obtained. The page dwell time threshold refers to the critical value for judging whether the user's dwell time on a page or operation step is an abnormal behavior. Its specific value can be dynamically set and adjusted according to the actual business scenario, user group behavior baseline, page function and expected operation path, and specific optimization goals and abnormal definitions.

[0087] In an embodiment, the time interval from browsing the home page to clicking the information detail page of user A is 90 seconds, and the time interval from clicking the information detail page to collecting the information is 30 seconds. When counting the number of clicks and interaction frequency of operation types, it can be found that user A has 10 clicks on browsing operations and 2 content interaction operations, with an interaction frequency of 0.2 content interaction behaviors per minute. If the user stays on the information detail page for more than 300 seconds, it is marked as an abnormal stay. This abnormal detection helps to identify whether the user's stay time is too long due to page loading problems or content attractiveness, thereby optimizing page design or content recommendation strategy.

[0088] In step S1103, if the number of clicks in the interaction frequency feature set is less than the preset device interaction effectiveness threshold, the access path and session duration data are extracted and grouped to obtain the user access mode feature set.

[0089] It should be noted that the device interaction effectiveness threshold is a pre-set minimum number of clicks standard for judging whether the amount of user interaction data collected from a specific device is sufficient for effective mode analysis. It is based on the click frequency distribution of a specific device type in historical interaction data and is pre-set by statistical methods, for example, the 20th percentile or average value of the historical number of clicks of the device type. If the number of clicks of a certain device type in the interaction frequency feature set is less than the device interaction effectiveness threshold, the access path and session duration are further extracted. The K-means clustering algorithm is used for grouping, and the "access path" and "session duration" data extracted for each user are converted into numerical feature vectors; K data points are randomly selected as initial cluster centers, the distance of each data point to each cluster center is calculated, and it is assigned to the cluster where the nearest cluster center is located; the mean value of each cluster is recalculated, and the mean value is taken as the new cluster center; repeat the above steps until the cluster center no longer changes significantly or the preset number of iterations is reached, and the algorithm converges. All data points are divided into K clusters. Each cluster represents a group of users with similar access behavior, i.e. a user access mode feature.

[0090] In an embodiment, the number of clicks of user A through the mobile phone is 3, which is lower than the preset device interaction effectiveness threshold (5, set by the average value of historical click numbers), so the system determines that the behavior data on the device is sparse, and further triggers the extraction of the "access path" (home page -> search page -> content detail page) and "session duration" (5 minutes). After collecting enough similar data, the K-means clustering algorithm is used to group all similar low-interaction data. The algorithm can automatically divide users into modes such as fast browsing and deep searching. This accurate grouping helps to achieve differentiated services, such as directly pushing popular or high-attention content to fast browsing users to improve the efficiency of content services and user engagement.

[0091] In step S1104, the missing proportion of time intervals and operation types is counted for the user access mode feature set, and it is determined whether the missing data is concentrated in a specific device type or time interval, to obtain a data missing distribution feature set.

[0092] In an embodiment, when the missing proportion of time intervals and operation types is counted for the user access mode feature set, 10% of the access time stamps are missing and 15% of the operation types are not recorded in the data set. Analysis shows that the missing data is concentrated in mobile phone devices and night time periods, which may be caused by unstable mobile phone log recording or less user behavior at night. The data missing distribution feature set can help optimize the log collection system and prioritize the repair of mobile phone data recording problems. This analysis not only improves data integrity, but also provides a reliable data foundation for subsequent modeling.

[0093] In step S12, according to the data missing distribution, boundary analysis is performed on the surrounding complete data, and boundary features are extracted, including:

[0094] S1201, based on the data missing distribution feature set, obtaining user location trajectory data and event association matrix data and identifying complete data boundaries to obtain a time connection point set;

[0095] S1202, according to the time connection point set, counting the distribution frequency of user activity events on location points to obtain a location connection point set;

[0096] S1203, for the location connection point set and the time connection point set, constructing a time-location association graph of user activities and grouping the nodes in the graph to obtain a boundary feature set;

[0097] S1204, from the boundary feature set, counting the proportion of time connection points and location connection points in the data missing distribution feature set, if the proportion is lower than the preset data missing distribution proportion threshold, then extracting supplementary features from the complete data to obtain the boundary features of user activities.

[0098] In step S1201, based on the data missing distribution feature set, the user location trajectory data and the event association matrix data are obtained, and the complete data boundary is identified, to obtain a time connection point set.

[0099] It should be noted that the user location trajectory data and the event association matrix data are obtained from the user behavior log database, the continuity feature of the time sequence is analyzed, the critical point of data missing is located, the calculated time interval is compared with the preset time continuity threshold, and the complete data region and the missing data region are distinguished. The time continuity threshold is a configurable critical value for judging whether the data is continuous and complete by analyzing the timestamp interval. The threshold can be optimized and adjusted according to the requirements of the user behavior continuity of the business scenario for different time periods or user activity modes.

[0100] In an embodiment, the user location trajectory data is extracted from the user behavior log, which can record the jump trajectory of the user on different pages or function modules, such as that user B jumps from the home page to the search page at 2025-09-08 14:00:00, enters the content detail page at 14:01:15, and accesses the personal center page at 14:03:00, forming the location trajectory “home page→search page→content detail page→personal center”. The time sequence data set is generated based on the timestamp sorting to ensure that the trajectory is arranged in chronological order and reflects the user behavior continuity. In the behavior sequence of user B, if the timestamp interval exceeds the preset time continuity threshold of 600 seconds, it is considered that there is a data breakpoint, which is marked as a non-complete boundary. Assuming that user B accesses the personal center page at 14:03:00, and the next record is 14:15:00 to browse the home page, the interval is 720 seconds, which exceeds the time continuity threshold, and is marked as a time connection point, to generate a time connection point set. The user activity events are extracted from the event association matrix, and the distribution frequency of the events on the location point is counted.

[0101] In step S1202, according to the time connection point set, the distribution frequency of the user activity events on the location point is counted, to obtain a location connection point set.

[0102] It should be noted that according to the time connection point set, the corresponding user activity events are extracted from the event association matrix, the distribution frequency of the user activity events on the location point is counted, and if the distribution frequency is lower than the preset location interaction frequency threshold, it is determined to be abnormal, to obtain a location connection point set. The location interaction frequency threshold is a preset minimum standard for judging whether the frequency of the user activity events on a certain specific location point belongs to a normal range, which is mainly based on historical data statistics and quantile analysis.

[0103] In an embodiment, the event association matrix records that the user B has a browsing, commenting, etc. frequency of 0.5 times per minute on the content detail page. This frequency is higher than the preset location interaction frequency threshold (0.3 times per minute), and thus is determined to be normal. While the interaction frequency of the user B on the message center is only 0.1 times per minute, which is lower than the threshold, and thus the message center is determined to be an abnormal location point and is included in the location connection point set. This analysis can effectively find the "break points" or "resistance points" in the user interaction process, and provide accurate data insights for optimizing page design and user experience.

[0104] In step S1203, a time-location association graph of user activities is constructed for the location connection point set and the time connection point set, and the nodes in the graph are grouped to obtain a boundary feature set.

[0105] It should be noted that the time-location association graph of user activities is constructed for the location connection point set and the time connection point set, the nodes in the graph are grouped by using a K-means clustering algorithm, each node in the "time-location association graph" is converted into a multi-dimensional feature vector that integrates time, location and graph structure information, and the selection and iteration of cluster centers are performed, each cluster represents a user mode with similar "boundary behavior", and the cluster set constitutes the final boundary feature set.

[0106] In an embodiment, the association graph of the user B shows that the time node 14:00:00 to 14:03:00 corresponds to the location node "home page -> search page -> content detail page". By using the K-means clustering algorithm to group the nodes of the association graph, the user can be divided into "target-oriented type" and "deep exploration type". The target-oriented user has a short path and directly goes from the search page to the target content page; the deep exploration type user has a long path and will visit the search page and different content detail pages multiple times for browsing and comparison. After grouping, the boundary feature set is formed, reflecting the user behavior mode.

[0107] In step S1204, the proportion of the time connection point and the location connection point in the data missing distribution feature set is counted from the boundary feature set, and if the proportion is lower than a preset data missing distribution proportion threshold, a supplementary feature is extracted from the complete data to obtain the boundary feature of user activities.

[0108] It should be noted that the data missing distribution proportion threshold is a configurable critical value for measuring whether the discovered boundary feature is sufficient to represent the data missing situation. The threshold is usually set based on the analysis of the statistical distribution of the boundary feature coverage in the historical data (for example, the 10th percentile), and can be flexibly configured according to the requirement of data integrity.

[0109] In an embodiment, statistics show that the missing area covered by the current boundary feature set only accounts for 8% of the entire data missing distribution feature. The value is lower than the preset data missing distribution proportion threshold (10%). The system therefore determines that the existing boundary feature is not representative enough, and then triggers the supplement mechanism: new features such as "device type = tablet" and "session duration = 8 minutes" are extracted from the complete data and supplemented to the boundary feature set to enhance its representativeness. Conversely, if it is found that the "night mobile terminal location trajectory missing" mode accounts for 12% of the total missing (i.e., the proportion > 10%), it not only indicates that the existing boundary set needs to be supplemented, but more importantly, it directly locates a major missing source, providing a clear decision direction for optimizing the log collection system.

[0110] In step S13, if the association degree of the boundary feature with the surrounding complete data is higher than the preset association strength significance threshold, the surrounding data is grouped by anomaly mode detection to obtain the user behavior distribution rule, including:

[0111] S1301, based on the boundary feature, calculating the association strength of time points and location points in the ordered data set to obtain an association strength set;

[0112] S1302, if the association strength of a time point and a location point in the association strength set is higher than the preset association strength significance threshold, the corresponding user activity event is extracted and grouped to obtain an abnormal event set;

[0113] S1303, according to the abnormal event set, the frequency distribution of the user activity event in the time sequence and the location trajectory is counted and grouped to obtain a user behavior distribution rule set;

[0114] S1304, for the user behavior distribution rule set, the distribution proportion of the distribution rule in the time sequence and the location trajectory is calculated, and if the distribution proportion is lower than the preset distribution proportion threshold, a supplementary event feature is extracted from the complete data to obtain the final user behavior distribution rule.

[0115] In step S1301, based on the boundary feature, the association strength of time points and location points in the ordered data set is calculated to obtain an association strength set.

[0116] It should be noted that the time sequence data and the location trajectory data are obtained from the user behavior log database, an ordered data set is generated based on the timestamp sorting, a sliding window is used with a fixed window size, and the window is sequentially slid on the time axis, so that the continuous data stream is divided into a series of continuous, possibly overlapping local data blocks. The association strength of time points and location points in the ordered data set is calculated for the data in each window to obtain an association strength set.

[0117] In an embodiment, in the information service platform user behavior analysis scenario, time series data and location trajectory data are obtained from the user behavior log database, and ordered data sets are generated based on timestamp sorting, laying the foundation for subsequent analysis. The time series data records the time points of user operations, such as user C browsing the home page at 2025-09-08 15:00:00, entering the search page at 15:01:30, and accessing the content detail page at 15:02:45. The association strength between time points and location points is calculated using a sliding window. The sliding window is set to a fixed time range, such as 300 seconds, and the association frequency of time points and location points within the analysis window is analyzed. User C in the window from 15:00:00 to 15:05:00, the jump from the home page to the search page occurs 3 times, and the jump from the search page to the content detail page occurs 2 times. The association strength is calculated as the number of jumps divided by the total number of window times. If the association strength from the home page to the search page is 0.6, which is higher than the preset threshold of 0.5, it is considered that the jump mode has strong association, and an association strength set is generated.

[0118] In step S1302, if the association strength of a certain time point and location point in the association strength set is higher than the preset association strength significance threshold, the corresponding user activity event is extracted and grouped to obtain an abnormal event set.

[0119] It should be noted that the preset association strength significance threshold is a critical value for judging whether the association between behavior elements is significant. This threshold can be configured according to actual business scenarios and data analysis requirements, and is usually set based on the statistical distribution of historical data such as percentiles. If the association strength of a certain time point and location point in the association strength set is higher than the preset association strength significance threshold, the corresponding user activity event is extracted from the event matrix, and abnormal pattern detection is used to group the user activity event to obtain an abnormal event set.

[0120] In an embodiment, the platform calculates the distribution of "time point-location point" association strength values by analyzing all user historical behavior data. According to business requirements, the association strength significance threshold can be set to the 95th percentile of the historical data distribution, for example, the corresponding value is 0.5. The association strength of user C jumping from "home page" to "search page" is 0.6, which is higher than the set association strength significance threshold, so the system extracts the detailed events of user C on the "search page", and finds that the "input event frequency" is 0.2 times per minute through abnormal detection, which is significantly lower than the normal range (0.4-0.8 times / minute), so this behavior is classified into the abnormal event set.

[0121] In step S1303, according to the abnormal event set, the frequency distribution of the user activity event in the time series and the location trajectory is counted and grouped to obtain a user behavior distribution rule set.

[0122] It should be noted that according to the set of abnormal events, the frequency distribution of the user activity events in the time sequence and the location trajectory is counted, and the K-means clustering algorithm is used to group the frequency distribution to obtain a set of user behavior distribution rules.

[0123] In an embodiment, the abnormal events of user C are mainly concentrated in the search page, with a frequency of 0.2 times per minute, and the content detail page is 0.5 times. By using the K-means clustering algorithm, the user behavior is divided into "low interaction type" and "high interaction type". The low interaction type user stays in the search page for a short time and has less interaction; the high interaction type user clicks frequently in the content detail page. After clustering, a set of user behavior distribution rules is formed, reflecting the behavior patterns of different user groups.

[0124] In step S1304, for the set of user behavior distribution rules, the distribution proportion of the distribution rule in the time sequence and the location trajectory is calculated, and if the distribution proportion is lower than a preset distribution proportion threshold, a supplementary event feature is extracted from the complete data to obtain a final user behavior distribution rule.

[0125] It should be noted that the distribution proportion threshold is a configurable critical value for measuring whether a certain user behavior distribution rule has sufficient statistical significance and representativeness in the entire user group or data set. The threshold can be set according to the business requirements for rule reliability, historical distribution proportion, and other factors, and can be iteratively optimized according to the actual application effect.

[0126] In an embodiment, the distribution proportion of the distribution rule in the time sequence and the location trajectory is calculated. For example, the search page abnormal event proportion of the low interaction type user is 15%, which is lower than the preset distribution proportion threshold of 20%, and the supplementary event feature is extracted from the complete data, such as the device type of user C is a mobile phone, the session duration is 5 minutes, and the final user behavior distribution rule is added. The extraction of such supplementary features enriches the dimensions of user behavior analysis and provides data support for subsequent optimization.

[0127] In step S14, according to the user behavior distribution rule, the potential associated path of the data missing area is extracted, and a judgment is made to obtain a user behavior pattern sketch, including:

[0128] S1401, based on the user behavior distribution rule, if the association strength of a certain time point and location point in the association strength set is lower than a preset association strength significance threshold, the data missing area is extracted, and a set of potential paths is identified and obtained;

[0129] S1402, according to the set of potential paths, the event frequency distribution is calculated and grouped to obtain a set of user activity classifications;

[0130] S1403, for the user activity classification set, combine the time sequence characteristics and the location trajectory association to construct a behavior pattern framework, and determine a user mode sketch.

[0131] In step S1401, based on the user behavior distribution rule, if the association strength of a time point and a location point in the association strength set is lower than a preset association strength significance threshold, a data missing area is extracted, and a potential path set is identified and obtained.

[0132] It should be noted that if the association strength of a time point and a location point in the association strength set is lower than a preset threshold, a data missing area is extracted from the ordered data set, and the most likely existing potential association path is intelligently inferred according to the existing user behavior rule. The specific operation is as follows: a user behavior graph is constructed, each node represents a location point, and each directed edge represents a possible jump relationship; the starting location point of the data missing area and the location point where the user appears again after the end of the data missing area are defined, and a depth-first search is used to set a potential path set. Each path is a reasonable assumption of the possible behavior of the user in the data missing area.

[0133] In one embodiment, user D has a jump from the home page to the classification page twice and a jump from the classification page to the favorite list once in the window of 10:00:00 to 10:03:00. The association strength is the number of jumps divided by the total number of windows. If the association strength of the jump from the home page to the classification page is 0.4, which is lower than the preset association strength significance threshold of 0.5, it is considered that the association of the jump mode is weak, and the data missing area needs to be further analyzed. For the area with an association strength lower than the threshold, a data missing area is extracted from the ordered data set, and a graph traversal algorithm is used to identify a potential association path. User D has a data missing from the classification page to the favorite list, which may be due to the user not completing the jump or the data not being recorded. The graph traversal algorithm takes the page as the node and the jump as the edge to construct a user behavior graph and search for a potential path from the classification page to the favorite list, such as "classification page → content detail page → favorite list".

[0134] In step S1402, according to the potential path set, the event frequency distribution is calculated and grouped to obtain a user activity classification set.

[0135] It should be noted that according to the potential path set and the historical activity sequence, the event frequency distribution is calculated, the K-means clustering algorithm is used to group the frequency distribution, and a user activity classification set is obtained.

[0136] In an embodiment, the user D has a browsing event frequency of 0.3 times per minute on the category page and 0.6 times on the collection list page. Based on these frequencies, the user behavior is classified into "browsing dominant type" and "content focus type" using the K-means clustering algorithm. The browsing dominant type user has a long stay time on the category page and browses frequently; the content focus type user interacts more on the collection list page. After clustering, a user activity classification set is formed, clearly distinguishing different behavior patterns.

[0137] In step S1403, for the user activity classification set, a behavior pattern framework is constructed in combination with the time series features and location trajectory association, and a user pattern sketch is determined.

[0138] It should be noted that for the user activity classification set, a behavior pattern framework is constructed in combination with the time series features and location trajectory association, and a behavior pattern sketch is determined by comparing a preset behavior pattern sketch template. The preset behavior pattern sketch template is one or more predefined and standardized feature rule sets, which are mainly established based on historical data clustering and statistical modeling.

[0139] In an embodiment, for the user activity classification set, a behavior pattern framework is constructed in combination with the time series features and location trajectory association. The browsing dominant behavior of user D is characterized by a category page stay time of more than 2 minutes and a high click frequency. By comparing the preset behavior pattern sketch template, such as the "browsing dominant type template" requiring a category page stay time of more than 1.5 minutes and a click frequency of more than 0.2 times per minute, it is determined that the behavior of user D conforms to the template. This framework provides data support for subsequent personalized recommendation.

[0140] In step S15, based on the user behavior pattern sketch, the user behavior distribution law is propagated to the data missing area to obtain a missing data inference value, including:

[0141] S1501, based on the user behavior pattern sketch as prior knowledge, the statistical pattern in the user behavior distribution law set is taken as label information, and is propagated to the unlabeled node corresponding to the data missing area through the edge in the time-location association graph;

[0142] S1502, according to the distribution probability converged on the unlabeled node, taking the activity event corresponding to the maximum probability as the missing data inference value of the data missing area.

[0143] In step S1501, based on the user behavior pattern sketch as prior knowledge, the statistical pattern in the user behavior distribution law set is taken as label information, and is propagated to the unlabeled node corresponding to the data missing area through the edge in the time-location association graph.

[0144] It should be noted that the label probability distribution of the marked nodes in the graph is initialized according to the user behavior pattern sketch and the behavior distribution law obtained from the complete data. The weight of the edge in the graph is defined based on the frequency of behavior transition between nodes or the similarity of time intervals. The higher the transition frequency or the more similar the time intervals, the greater the weight. Then the label probability distribution of the unmarked nodes is updated according to the labels of the neighbor nodes and the edge weight through iteration. The stop condition of iteration is set to the maximum iteration number, such as 100, when the change difference of the label probability distribution of all nodes between two adjacent iterations reaches the maximum iteration number. Through this mechanism, even in the case of missing data of some nodes, the most likely behavior pattern can be inferred based on the association relationship with the known nodes.

[0145] In an embodiment, the jump data of user E from the "collection list" to the "personal center" is missing. The system first initializes the label probability distribution of the marked nodes in the graph based on the behavior pattern sketch that user E has been identified as a "content preference type", and the distribution law obtained from the complete data that "70% of the content preference type users will first access the recommended content page". Then, in the time-location association graph, the system takes the "collection list" node as the starting point, and through its connection edge, the label information that "content preference type users often access recommended content" is propagated and diffused to the surrounding unmarked missing area, providing a basis for subsequent probability calculation.

[0146] In step S1502, the activity event corresponding to the maximum probability is taken as the missing data inference value of the data missing area according to the converged distribution probability on the unmarked node.

[0147] It should be noted that the label propagation is an iterative process, and when the label probability distribution on all nodes no longer changes significantly, it is considered to be converged. At this time, each unmarked node will obtain a probability distribution vector about all possible behavior labels. Taking the activity event corresponding to the maximum probability is a decision rule based on the maximum posterior probability, which selects the most likely event to occur under the given network structure and existing label information as the inference result. This method can ensure that the inference result is the most consistent in the global relationship network, rather than relying only on local information.

[0148] In an embodiment, after multiple rounds of iteration calculation, the probability distribution converged on the unmarked node corresponding to the data missing area of user E is: {accessing the recommended content page: 0.7, directly accessing the personal center: 0.25, returning to the content detail page: 0.05}. According to the principle of taking the maximum probability, the system selects accessing the recommended content page as the missing data inference value of the data missing area. This shows that the system infers that user E has a 70% probability of missing data recording in the subsequent path due to jumping to the recommended content page, thereby completing the user behavior sequence and providing key data support for analyzing the breakpoints in the user behavior path.

[0149] In step S16, the potential correlation path is integrated according to the missing data inference value, if the missing data inference value matches the boundary feature, the surrounding complete data is fused, and the internal structure of the data missing area is determined, including:

[0150] S1601, the missing data inference value is filled into the potential path set to generate a completed candidate path set;

[0151] S1602, the matching degree of the candidate path set and the boundary feature set is calculated, if the matching degree is higher than the preset matching threshold, it is determined that the matching path boundary consistency is consistent;

[0152] S1603, based on the matching path boundary consistency, the correlation features of the surrounding complete data are fused, the weighted value of each feature point in the data missing area is calculated according to the matching degree as the weight, and the internal structure of the data missing area is determined.

[0153] In step S1601, the missing data inference value is filled into the potential path set to generate a completed candidate path set.

[0154] It should be noted that the preliminary reconstruction of the data missing area is realized by the path completion operation, the missing data inference value obtained by the path selection mechanism based on probability is used as the selection basis, the multiple potential correlation paths obtained by graph traversal are weighted and integrated, a comprehensive score is calculated for each potential path, and the paths with scores higher than the preset threshold are retained to form a completed candidate path set. This process ensures that the finally generated path set not only contains all possible behavior trends, but also highlights the most likely path through the probability weight, which lays a solid foundation for subsequent matching verification.

[0155] In an embodiment, the missing data inference value set of the user F indicates that it has an 80% probability of selecting the "collection list→detail confirmation page→personal center" path, and a 20% probability of selecting the "collection list→content interaction page→personal center" path. The system fills these two paths and their corresponding probability values into the initial potential path set to generate a weighted candidate path set. This set not only records the path sequence, but also marks the likelihood value of each path, forming a complete behavior path hypothesis space with confidence.

[0156] In step S1602, the matching degree of the candidate path set and the boundary feature set is calculated, if the matching degree is higher than the preset matching threshold, it is determined that the matching path boundary consistency is consistent.

[0157] It should be noted that the matching degree of the candidate path set and the boundary feature set is calculated by using a sequence similarity-based calculation method such as dynamic time warping (DTW). The Euclidean distance between each point of one sequence and each point of another sequence is calculated to form a distance matrix. A path with the minimum accumulated distance is found from the lower left corner to the upper right corner of the matrix. The path defines the best point-to-point matching manner between the two sequences. The distances of all points on the optimal path are added to obtain the final accumulated distance, and the accumulated distance is normalized. The candidate path is compared with the boundary feature set. The matching degree is a value between 0 and 1, reflecting the extent to which the candidate path can naturally connect the known start and end boundaries. The preset matching threshold is a critical value for judging whether the matching degree calculated by DTW is high enough to accept the candidate path. It is usually set based on historical verification data (for example, 0.7), and can be configured differently for different path complexity or business criticality. If the matching degree is higher than the threshold, it indicates that the completed path is highly consistent with the context, and is determined to have path boundary consistency.

[0158] In an embodiment, the boundary feature set of the user F includes two reliable data points of entering the collection list and finally appearing in the personal center. The system calculates the matching degree of the candidate path "collection list->detail confirmation page->personal center" of the user F and the boundary. Since the path perfectly connects the two boundary points, and the detail confirmation page is a reasonable step before accessing the personal center, the matching degree is calculated as 0.95, which is much higher than the preset matching threshold 0.7. Therefore, the system determines that the path matches the boundary feature and has high path boundary consistency.

[0159] In step S1603, based on the matched path boundary consistency, the associated features of the surrounding complete data are fused, the weighted values of the feature points in the data missing area are calculated based on the matching degree as the weight, and the internal structure of the data missing area is determined.

[0160] It should be noted that the associated features of the surrounding complete data refer to the statistical features exhibited by the complete data adjacent to the missing area in space and time. The calculated matching degree is used as the weight, and the feature values implied by the candidate path and the associated features of the surrounding complete data are weighted and averaged. The higher the matching degree, the greater the weight of the features of the candidate path in the final fusion result. By calculating the weighted values of each feature point in the missing area, a quantitative internal structure representation of the missing part logical framework is finally generated.

[0161] In one embodiment, for the data missing area of user F, the surrounding complete data shows that the average detail confirmation page stay duration of the user at the site is 45 seconds. The matching degree of the candidate path of user F is 0.95, and the system fuses the feature of "staying at the detail confirmation page" implied by the path with the average stay duration of the surrounding data as the weight. The weighted stay duration of user F at the detail confirmation page is finally determined as the product of 0.95 and the candidate path inference duration and the sum of 0.05 and the average duration. In this way, the system not only infers where the user went, but also generates the possible behavior intensity of the user there, and finally forms a complete and quantitative internal structure set, clearly revealing the logical framework of the data missing area.

[0162] In step S17, a user activity sequence is generated by correlation weight analysis according to the internal structure of the data missing area and the missing data inference value, including:

[0163] S1701, based on the internal structure of the missing area and the missing data inference value, a time series reorganization data set sorted and spliced by timestamp is obtained;

[0164] S1702, based on the time series reorganization data set, the transfer rule of activity events in time is analyzed, and the most possible user activity sequence is decoded to obtain an initial user activity sequence;

[0165] S1703, the initial user activity sequence is subjected to smoothing filtering processing to generate a final user activity sequence.

[0166] In step S1701, based on the internal structure of the missing area and the missing data inference value, a time series reorganization data set sorted and spliced by timestamp is obtained.

[0167] It should be noted that the feature points extracted from the internal structure of the data missing area are the specific events or page identifiers obtained from the missing data inference value. According to the timestamp information provided by the internal structure and the event content provided by the inference value, the filled data points are inserted into the corresponding position of the original ordered data set in timestamp order, thereby forming a time series reorganization data set that is continuous and complete in time dimension. This process reconstructs the time sequence logic of user behavior, providing a high-quality data basis for subsequent sequence analysis.

[0168] In one embodiment, the original data of user H is missing after "event page". The internal structure set infers that the user can "return to content detail page" at 15:03:30 and stay for 90 seconds, and the missing data inference value indicates that the event is "browsing content detail". The system fuses these two pieces of information, generates a data point with timestamp 15:03:30 and event [Location: content detail page, Action: browse, Duration: 90s], and inserts it into the original sequence after "event page (15:03:00)" in chronological order, thereby forming a time series reorganized data set containing home page→content detail page→favorite page→event page→[inferred] content detail page.

[0169] In step S1702, based on the time series reorganized data set, the transfer rule of the activity event in time is analyzed, and the most likely user activity sequence is decoded to obtain an initial user activity sequence.

[0170] It should be noted that this step aims to extract the most likely and globally optimal user behavior sequence from the reorganized data. The core is to use sequence modeling and decoding algorithms such as hidden Markov model or conditional random field. These models regard each position point in the reorganized data set as a state, and by learning and analyzing the transition probability between states, i.e., the regularity of user jumping from the previous page to the next page and the occurrence probability of observed events, a sequence probability model is constructed. Subsequently, a decoding algorithm such as Viterbi algorithm is used to find a path with the highest global probability in the entire sequence probability model, and this path is the decoded most likely user activity sequence, i.e., the initial user activity sequence.

[0171] In one embodiment, based on the time series reorganized data set of user H, the system uses a hidden Markov model for training. The model finds that the probability of transitioning from "event page" to "content detail page" is relatively high (0.7), while the probability of directly transitioning from "event page" to "personal center" is relatively low (0.1). Combined with the long stay time (observed event) of user H in "content detail page", the Viterbi algorithm decodes the globally optimal path: home page→content detail page→favorite page→event page→content detail page. This path is output as the initial user activity sequence, which best fits all observed and inferred user behavior rules.

[0172] In step S1703, the initial user activity sequence is subjected to smoothing filtering processing to generate a final user activity sequence.

[0173] It should be noted that the initial user activity sequence can contain isolated abnormal points or unreasonable short jumps caused by inference errors or data noise. The purpose of the smoothing filtering process is to improve the rationality and fluency of the sequence. The present application adopts a method combining rule-based filtering and statistical moving window smoothing, defines a minimum reasonable dwell time threshold and a reasonable jump rule library. For page points with too short dwell time in the sequence, they will be considered as noise and merged or deleted; for transitions that do not conform to common jump logic, such as suddenly jumping from a function page to the home page, they will be adjusted according to the rule library and the context information of the surrounding window, and replaced or inserted with more reasonable pages, so as to output a final user activity sequence that is closer to the real user behavior pattern and smooth and coherent.

[0174] In an embodiment, there is an inference point in the initial sequence of user H: after the activity page, it jumps to the content detail page only after 2 seconds. The system queries that the average loading time of the "activity page" is 1.5 seconds, and the 10th percentile of the historical dwell time distribution is 8 seconds. Therefore, the system dynamically calculates that the reasonable minimum dwell time threshold of the page is about 5 seconds, judges that the dwell time of 2 seconds is lower than the minimum reasonable dwell time threshold, and considers that the record can be unreasonable noise or inference deviation. Combined with the interest preference of the user such as "like to understand the content in depth", the smoothing filter corrects the short jump to a reasonable 90-second dwell, generates a more smooth and reliable final user activity sequence, and provides accurate data input for subsequent user portrait generation. Among them, the minimum reasonable dwell time threshold is a critical time value for judging whether the user's dwell time on a certain interface is too short to be unreasonable or can be caused by noise / error, which is mainly based on historical data statistics and distribution analysis.

[0175] In step S18, according to the user activity sequence, the consistency of the internal structure of the data missing area and the missing data inference value is verified, and a complete user feature model is obtained, including:

[0176] S1801, based on the user activity sequence, extracting the behavior segment of the area filled by the missing data inference value as a to-be-verified sequence segment;

[0177] S1802, calculating the behavior feature matching degree between the to-be-verified sequence segment and the missing data inference value, and checking the logical coherence with the boundary feature;

[0178] S1803, if the behavior feature matching degree is higher than a preset consistency threshold, and the logical coherence test passes, it is determined that the internal structure of the data missing area is consistent with the missing data inference value, and a consistency determination result is obtained;

[0179] S1804, based on the consistency determination result, the user activity sequence is fused with the interest label to generate a complete user feature model.

[0180] In step S1801, based on the user activity sequence, the behavior segment of the filled region by the missing data inferred value is extracted as a to-be-verified sequence segment.

[0181] It should be noted that the to-be-verified sequence segment refers to a continuous behavior segment in the user activity sequence that originally has data missing but has been filled through the foregoing steps. When extracting, the start and end positions of the filled region are located according to the identification information in the sequence, so as to intercept the corresponding subsequence. The subsequence includes the inferred user behavior and is a direct object of subsequent consistency verification.

[0182] In an embodiment, the initial activity sequence of user K has data missing after the "detail confirmation page", and after being filled by inference, a complete sequence including "detail confirmation page → content detail page → personal center" is generated. According to the "inferred filling" identification in the sequence, the system extracts "content detail page → personal center" as a to-be-verified sequence segment for subsequent verification of whether the inferred segment is reasonable.

[0183] In step S1802, the behavior feature matching degree between the to-be-verified sequence segment and the missing data inferred value is calculated, and the logical coherence with the boundary feature is checked.

[0184] It should be noted that the behavior feature matching degree is calculated by comparing the similarity between the actual behavior feature in the to-be-verified sequence segment and the predicted behavior feature in the missing data inferred value. Cosine similarity or other correlation indicators can be used for quantification. At the same time, the logical coherence check is to evaluate whether the to-be-verified sequence segment is consistent with the user behavior pattern reflected by the previously extracted boundary feature, for example, to check whether the behavior in this segment is within the reasonable time window or common path range defined by the boundary feature.

[0185] In an embodiment, for the to-be-verified sequence segment "content detail page → personal center" of user K, the system calculates the matching degree of the actual stay duration of "content detail page" in this segment, such as 60 seconds, with the predicted stay duration in the inferred value, such as 55 seconds, such as 90%. At the same time, it is checked whether the jump from the content detail page to the personal center conforms to the rule that "the detail confirmation page is usually strongly associated with the personal center" in the boundary feature set, thereby completing the logical coherence test.

[0186] In step S1803, if the behavior feature matching degree is higher than a preset consistency threshold, and the logical coherence test passes, it is determined that the internal structure of the data missing region is consistent with the missing data inferred value, and a consistency determination result is obtained.

[0187] It should be noted that the preset consistency threshold is a configurable critical value for judging whether the matching degree of the behavior characteristics of the to-be-verified sequence segment and the missing data inference value is acceptable. The threshold is usually set based on the matching degree distribution of historical correct inference cases, for example, the 85th percentile, and can be dynamically adjusted according to the business requirements for the accuracy of the portrait. The passing standard of the logical coherence test can include that the to-be-verified sequence segment does not violate the constraint conditions of the boundary characteristic definition. Only when both the high matching degree and the logical coherence are met, the inference is considered reliable, and the internal structure of the data missing area is correctly reconstructed.

[0188] In an embodiment, the behavior characteristic matching degree of the to-be-verified sequence segment of user K is 90%, which is higher than the consistency threshold 70%, and its jump path conforms to the common mode of the boundary characteristic definition, so the system determines that the internal structure of the data missing area is consistent with the missing data inference value, and generates a consistency determination result.

[0189] In step S1804, based on the consistency determination result, the user activity sequence and the interest label are fused to generate a complete user feature model.

[0190] It should be noted that when the consistency determination result is "consistent", it indicates that the filled user activity sequence is reliable and continuous. At this time, the sequence is fused with interest labels such as "content preference type" and "deep browsing type" in multiple dimensions. The fusion method can include a weighted average integration strategy, and finally form a comprehensive, complete, and dynamic behavior and interest label containing user feature model for precise recommendation, clustering analysis, and other downstream applications.

[0191] In an embodiment, the consistency determination result of user K is "consistent", and the system fuses its complete user activity sequence containing the inferred and filled "content detail page -> personal center" segment with interest labels such as "deep browsing type" in the feature layer to generate a multi-dimensional user feature vector, which constitutes a complete user feature model and is stored in the user portrait database for subsequent use.

[0192] In summary, the application discloses a user portrait generation method based on big data, which comprises extracting time series and interaction frequency features from user behavior data, constructing user access mode feature set and analyzing data missing distribution; extracting boundary features based on missing distribution, if its association with complete data is significant, detecting surrounding data to identify behavior rules; extracting potential association path in missing area according to the behavior rules, inferring user behavior mode prototype; propagating behavior rules to missing area to generate data inference value; fusing path and boundary features to determine internal structure of missing area and generate user activity sequence; verifying consistency of internal structure and inference value to form complete user feature model. The application realizes deep mining and accurate inference of context in missing area of user behavior data through boundary analysis, association strength threshold control, semi-supervised learning propagation, path-boundary consistency matching and sequence continuity representation, effectively improves the accuracy, completeness and behavior logic consistency of user portrait, and provides reliable basis for personalized recommendation and user behavior analysis.

[0193] Reference Figure 2 The second embodiment of the application provides a user portrait generation system based on big data, comprising:

[0194] A data acquisition and preprocessing module acquires user behavior raw data, extracts time series features and interaction frequency features from the user behavior raw data and analyzes them to determine data missing distribution.

[0195] A boundary feature extraction module performs boundary analysis on surrounding complete data according to the data missing distribution and extracts boundary features.

[0196] A behavior rule discovery module detects surrounding data by abnormal mode if the association degree of the boundary features and the surrounding complete data is higher than a preset association strength significance threshold, and obtains user behavior distribution rules.

[0197] A mode prototype identification module extracts potential association paths in the data missing area according to the user behavior distribution rules and judges to obtain a user behavior mode prototype.

[0198] A missing data inference module propagates the user behavior distribution rules to the data missing area based on the user behavior mode prototype to obtain missing data inference values.

[0199] An internal structure construction module integrates the potential association paths according to the missing data inference values, fuses the surrounding complete data if the missing data inference values match the boundary features, and determines the internal structure of the data missing area.

[0200] a user activity sequence generation module, which generates a user activity sequence by correlation weight analysis according to the internal structure of the data missing area and the missing data inference value;

[0201] a consistency verification and model generation module, which verifies the consistency of the internal structure of the data missing area and the missing data inference value, and obtains a complete user feature model.

[0202] It should be noted that the user portrait generation system based on big data provided by the embodiments of the present application is used to execute all process steps of the user portrait generation method based on big data provided by the above embodiments, and the working principles and beneficial effects of the two are one-to-one correspondence, thus not being repeated.

[0203] The embodiments of the present application also provide an electronic device. The electronic device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a user portrait generation method based on big data program. The processor implements the steps in the above various user portrait generation method embodiments based on big data when executing the computer program, such as Figure 1 the step S11 shown. Alternatively, the processor implements the functions of each module / unit in the above various device embodiments when executing the computer program, such as the missing data inference module.

[0204] For example, the computer program can be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program in the electronic device.

[0205] The electronic device can be a desktop computer, a notebook, a palm computer, and a smart tablet, etc. The electronic device can include, but is not limited to, a processor, a memory. Those skilled in the art can understand that the above components are only examples of the electronic device, and do not constitute a limitation on the electronic device, and can include more or fewer components than the above, or combine certain components, or different components, for example, the electronic device can also include an input / output device, a network access device, a bus, etc.

[0206] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The processor is a control center of the electronic device, and connects various parts of the electronic device through various interfaces and lines.

[0207] The memory can be used to store the computer program and / or modules, and the processor realizes various functions of the electronic device by running or executing the computer program and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function, an image playing function, etc.), etc. The data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a nonvolatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state memory device.

[0208] The modules / units integrated in the electronic device, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer readable medium can include any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. that can carry the computer program code. It should be noted that the contents included in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.

[0209] It should be noted that the above-described device embodiments are only schematic, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. In addition, the connection relationship between the modules in the device embodiment provided by the present application indicates that there is a communication connection between them, which can be realized as one or more communication buses or signal lines. Those skilled in the art can understand and implement it without creative labor.

[0210] The above-described specific embodiments further illustrate the purpose, technical solutions, and beneficial effects of the present application. It should be understood that the above-described specific embodiments are only examples of the present application and are not intended to limit the protection scope of the present application. In particular, any modifications, equivalent replacements, improvements, etc. made by those skilled in the art within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1.A big data-based user profiling method, characterized by, The method comprises the following steps: acquiring user behavior raw data, extracting time sequence features and interaction frequency features from the user behavior raw data, and analyzing and determining a data missing distribution; performing boundary analysis on surrounding complete data according to the data missing distribution, and extracting boundary features; if the correlation degree of the boundary features and the surrounding complete data is higher than a preset correlation strength significance threshold, grouping surrounding data through abnormal mode detection to obtain a user behavior distribution rule; extracting a potential correlation path of a data missing area according to the user behavior distribution rule, and judging to obtain a user behavior mode sketch; based on the user behavior mode sketch, propagating the user behavior distribution rule to the data missing area to obtain a missing data inference value; integrating the potential correlation path according to the missing data inference value, and if the missing data inference value matches the boundary features, fusing the surrounding complete data to determine an internal structure of the data missing area; generating a user activity sequence through correlation weight analysis according to the internal structure of the data missing area and the missing data inference value; verifying the consistency of the internal structure of the data missing area and the missing data inference value according to the user activity sequence, and obtaining a complete user feature model; wherein, according to the user behavior distribution rule, the potential correlation path of the data missing area is extracted, and a judgment is made to obtain a user behavior mode sketch, which comprises: based on the user behavior distribution rule, if the correlation strength of a certain time point and position point in the correlation strength set is lower than a preset correlation strength significance threshold, the data missing area is extracted, and a potential path set is identified and obtained; according to the potential path set, calculating the event frequency distribution and grouping to obtain a user activity classification set; for the user activity classification set, combining the time sequence features and location trajectory correlation to construct a behavior mode framework, and judging a user mode sketch. 2.The big data based user profiling method of claim 1, wherein, acquiring user behavior raw data, extracting time sequence features and interaction frequency features from the user behavior raw data, and analyzing and determining a data missing distribution, which comprises: acquiring user behavior raw data, generating a behavior sequence by timestamp sorting to obtain a user behavior time sequence data set; according to the user behavior time sequence data set, calculating the time interval of adjacent access times and counting the click number and interaction frequency to obtain an interaction frequency feature set; if the click number in the interaction frequency feature set is lower than a preset device interaction effectiveness threshold, extracting access path and session duration data and grouping to obtain a user access mode feature set; for the user access mode feature set, counting the missing proportion of time interval and operation type and judging whether the missing data is concentrated in a specific device type or time interval to obtain a data missing distribution feature set. 3.The big data based user profiling method of claim 2, wherein, according to the data missing distribution, performing boundary analysis on surrounding complete data, and extracting boundary features, which comprises: based on the data missing distribution feature set, acquiring user location trajectory data and event correlation matrix data, and identifying complete data boundaries to obtain a time connection point set; According to the time connection point set, the distribution frequency of the user activity event on the location point is counted to obtain a location connection point set; For the location connection point set and the time connection point set, a time-location association graph of the user activity is constructed, and nodes in the graph are grouped to obtain a boundary feature set; From the boundary feature set, the proportion of the time connection point and the location connection point in the data missing distribution feature set is counted, and if the proportion is lower than a preset data missing distribution proportion threshold, a supplementary feature is extracted from complete data to obtain a boundary feature of the user activity. 4.The big data based user profiling method of claim 3, wherein, If the association degree of the boundary feature and the surrounding complete data is higher than a preset association strength significance threshold, surrounding data is grouped through abnormal mode detection to obtain a user behavior distribution rule, including: Based on the boundary feature, the association strength of the time point and the location point in the ordered data set is calculated to obtain an association strength set; If the association strength of a certain time point and a location point in the association strength set is higher than a preset association strength significance threshold, the corresponding user activity event is extracted and grouped to obtain an abnormal event set; According to the abnormal event set, the frequency distribution of the user activity event in the time sequence and the location trajectory is counted and grouped to obtain a user behavior distribution rule set; For the user behavior distribution rule set, the distribution proportion of the distribution rule in the time sequence and the location trajectory is calculated, and if the distribution proportion is lower than a preset distribution proportion threshold, a supplementary event feature is extracted from complete data to obtain a final user behavior distribution rule. 5.The big data based user profiling method of claim 4, wherein, Based on the user behavior mode sketch, the user behavior distribution rule is propagated to the data missing area to obtain a missing data inference value, including: Based on the user behavior mode sketch as prior knowledge, the statistical mode in the user behavior distribution rule set is taken as label information, and is propagated to the unmarked node corresponding to the data missing area through the edge in the time-location association graph; According to the distribution probability converged on the unmarked node, the activity event corresponding to the maximum probability is taken as the missing data inference value of the data missing area. 6.The big data based user profiling method of claim 1, wherein, According to the missing data inference value, the potential association path is integrated, and if the missing data inference value matches the boundary feature, the surrounding complete data is fused to determine the internal structure of the data missing area, including: The missing data inference value is filled into the potential path set to generate a completed candidate path set; The matching degree of the candidate path set and the boundary feature set is calculated, and if the matching degree is higher than a preset matching threshold, it is determined that the matching path boundary consistency is consistent; Based on the matching path boundary consistency, the association features of the surrounding complete data are fused, the weighted values of each feature point in the data missing area are calculated by taking the matching degree as a weight, and the internal structure of the data missing area is determined. 7.The big data based user profiling method of claim 1, wherein, According to the internal structure of the data missing area and the missing data inference value, a user activity sequence is generated through association weight analysis, including: Based on the internal structure of the missing area and the missing data inference value, a time sequence reorganization data set sorted and spliced by time stamp is obtained; Based on the time sequence reorganization data set, the transfer rule of the active event in time is analyzed, the most possible user activity sequence is decoded, and an initial user activity sequence is obtained; The initial user activity sequence is subjected to smoothing filtering processing, and a final user activity sequence is generated. 8.The big data based user profiling method of claim 1, wherein, According to the user activity sequence, the consistency of the internal structure of the data missing area and the missing data inference value is verified, and a complete user feature model is obtained, including: Based on the user activity sequence, the behavior segment of the area filled by the missing data inference value is extracted as a to-be-verified sequence segment; The behavior feature matching degree between the to-be-verified sequence segment and the missing data inference value is calculated, and the logical coherence with the boundary feature is checked; If the behavior feature matching degree is higher than a preset consistency threshold, and the logical coherence check passes, it is determined that the internal structure of the data missing area is consistent with the missing data inference value, and a consistency determination result is obtained; Based on the consistency determination result, the user activity sequence and the interest label are fused to generate a complete user feature model. 9.A big data based user profiling system, characterized in that, A big data-based user portrait generation method according to any one of claims 1 to 8, comprising: A data acquisition and preprocessing module acquires user behavior raw data, extracts time sequence features and interaction frequency features from the user behavior raw data, and analyzes and determines data missing distribution; A boundary feature extraction module extracts boundary features by performing boundary analysis on surrounding complete data according to the data missing distribution; A behavior rule discovery module groups surrounding data through abnormal mode detection if the association degree of the boundary feature and the surrounding complete data is higher than a preset association strength significance threshold, and obtains user behavior distribution rules; A mode sketch identification module extracts a potential association path of a data missing area according to the user behavior distribution rules, and judges to obtain a user behavior mode sketch; A missing data inference module propagates the user behavior distribution rules to the data missing area based on the user behavior mode sketch, and obtains a missing data inference value; An internal structure construction module integrates the potential association path according to the missing data inference value, and fuses the surrounding complete data if the missing data inference value matches the boundary feature, to determine the internal structure of the data missing area; A user activity sequence generation module generates a user activity sequence by association weight analysis according to the internal structure of the data missing area and the missing data inference value; A consistency verification and model generation module verifies the consistency of the internal structure of the data missing area and the missing data inference value, and obtains a complete user feature model.

Images