A big data-based customer portrait key data mining platform and method

Abstract

The application belongs to the technical field of big data customer portrait technology, and discloses a customer portrait key data mining platform and method based on big data. The platform comprises: a behavior sequence acquisition module deployed on a user terminal, which interacts with a cloud module through a cloud collaborative mechanism; an intention drift degree evaluation module, which calculates the behavior type distribution difference between a current observation window and a historical benchmark window and generates an intention drift degree index; a sequence rearrangement module, which performs forward aggregation on high-intention behavior records to form an intention focus sequence when the index is higher than a threshold; a double-tower coding module, which codes full historical data and the intention focus sequence respectively; a weight control module, which dynamically generates a positive correlation fusion weight according to the intention drift degree index; and a fusion output module, which generates a customer portrait vector through weighted fusion. The application solves the technical problem of portrait response lag caused by the deterioration of short-term behavior sequence signal-to-noise ratio during intention drift in the prior art, and realizes the synchronous improvement of portrait accuracy and response speed.

Description

Technical Field

[0001] This invention relates to the field of big data customer profiling technology, and in particular to a key data mining platform and method for customer profiling based on big data. Background Technology

[0002] With the widespread application of big data technology in e-commerce, content recommendation, and precision marketing, customer profiling, as a core technical means to characterize user features and understand user intent, has become a fundamental capability for various internet platforms. Existing customer profiling methods typically collect user behavior data from multiple data sources, including browsing, clicking, adding to cart, searching, and saving records. Through data cleaning, feature engineering, and statistical modeling, a label system or vector representation representing long-term user preferences and short-term intents is generated. Building upon this, some existing technologies further introduce a dual-channel encoding architecture, independently encoding the user's full historical behavior data and recent real-time behavior data. Then, dynamic customer profiles are generated through attention mechanisms or weighted fusion to balance the stability of long-term user interests with the timeliness of short-term intents.

[0003] However, existing technologies generally suffer from a fundamental, long-overlooked technical flaw: when users experience intent drift during the consumption decision-making process—for example, quickly switching from casual browsing to a targeted purchase decision-making mode—the user's real-time behavior sequence becomes interspersed with a large number of transitional, low-intent behaviors. These low-intent behaviors coexist with a small number of high-intent behaviors on the timeline, causing the high-intent behavior evidence truly reflecting the user's current potential intent in the short-term encoder's input sequence to be diluted by a large amount of low-intent noise. Current technologies typically address this problem by adjusting the fusion weights of long-term and short-term features, failing to recognize the deeper root cause: the structural flaw in the short-term encoder's input sequence itself. Specifically, the deterioration of the signal-to-noise ratio in the sequence precedes the imbalance of the fusion weights. Therefore, simply adjusting the fusion weights cannot fundamentally solve the problem of delayed profile response and decreased accuracy during intent drift.

[0004] Therefore, this invention proposes a key data mining platform and method for customer profiling based on big data. Summary of the Invention

[0005] This invention provides a key data mining platform and method for customer profiling based on big data, which solves the fundamental defect in the prior art that the deterioration of the signal-to-noise ratio of short-term behavioral sequences during intent drift leads to lag in profiling response. Through closed-loop collaboration of intent drift-driven sequence rearrangement and adaptive dual-tower fusion, the accuracy of profiling and response speed are improved simultaneously.

[0006] This invention provides a key data mining platform for customer profiling based on big data, comprising: The behavior sequence acquisition module is used to collect user behavior data in real time and form raw behavior sequences sorted by time. The behavior sequence acquisition module is deployed on the user terminal device and interacts with other modules deployed in the cloud through a cloud collaboration mechanism. The intent drift assessment module is used to divide the original behavior sequence into multiple observation windows, calculate the difference in behavior type distribution between the current observation window and the historical baseline window, and generate an intent drift index based on the difference. The sequence rearrangement module is used to advance and aggregate the behavior records belonging to the preset high-intent behavior type in the original behavior sequence in the time dimension when the intent drift index is higher than the preset intent drift index threshold, so as to form an intent-focused sequence. The dual-tower encoding module includes a long-term interest encoder and a short-term intent encoder. The long-term interest encoder is used to encode the full historical behavior data to generate long-term interest vectors, and the short-term intent encoder is used to encode the intent focus sequence to generate short-term intent vectors. The weight control module is used to dynamically generate the fusion weight of long-term interest vector and short-term intention vector based on the intention drift index. The higher the intention drift index, the greater the fusion weight of the short-term intention vector. The fusion output module is used to weight and fuse long-term interest vectors and short-term intent vectors according to fusion weights to generate customer profile vectors.

[0007] Furthermore, the intent drift assessment module includes: The transition graph construction unit is used to construct the current transition graph of the current observation window and the historical transition graph of the historical benchmark window, respectively, with behavior type as the node, the previous behavior type pointing to the next behavior type of two consecutive behavior records in the original behavior sequence as the directed edge, and the occurrence number of consecutive behavior pairs as the weight of the directed edge. The in-degree decomposition unit is used to calculate the weighted in-degree and weighted out-degree of each behavior type node in the current transition graph. The weighted in-degree is the sum of the weights of all directed edges pointing to the current behavior type node, and the weighted out-degree is the sum of the weights of all directed edges originating from the current behavior type node. The transition bias calculation unit is used to calculate the rate of change of the weighted in-degree of each behavior type node in the current transition graph and the weighted in-degree of the historical transition graph, as well as the rate of change of the weighted out-degree of the current transition graph and the weighted out-degree of the historical transition graph. The difference calculation unit is used to perform a weighted summation of the in-degree change rate and out-degree change rate of all behavior type nodes. The weight is the normalized value of the frequency of occurrence of behavior type nodes in the current observation window. The weighted summation result is used as the difference in behavior type distribution between the current observation window and the historical baseline window. The index generation unit is used to perform a moving average of the differences across multiple consecutive observation windows to obtain the intention drift index.

[0008] Furthermore, the transfer bias calculation unit is also used for: Obtain the historical weighted out-degree and historical weighted in-degree of each behavior type node in the historical baseline window; For each behavior type node in the current transition graph, when the historical weighted out-degree is greater than zero, the ratio of the node's weighted out-degree in the current transition graph to the historical weighted out-degree is used as the out-degree anomaly coefficient; when the historical weighted out-degree is zero, the out-degree anomaly coefficient is set to a preset upper limit value. When the historical weighted in-degree is greater than zero, the ratio of the node's weighted in-degree in the current transition graph to the historical weighted in-degree is used as the in-degree anomaly coefficient; when the historical weighted in-degree is zero, the in-degree anomaly coefficient is set to a preset upper limit value. The product of the out-degree anomaly coefficient and the in-degree anomaly coefficient is used as the transition mode anomaly coefficient of the node. The difference calculation unit is also used to perform a weighted summation of the transition mode anomaly coefficients of all behavior type nodes, using the transition mode anomaly coefficient as the basis for weighted summation. The weight is the normalized value of the frequency of occurrence of behavior type nodes in the current observation window, and the weighted summation result is used as the difference in behavior type distribution between the current observation window and the historical baseline window.

[0009] Furthermore, the sequence rearrangement module includes: The causal association identification unit is used to trace back in the original behavior sequence for each target behavior record that belongs to the preset high-intent behavior type. It identifies behavior records that occurred before the target behavior record, belong to the preset preparatory behavior set, and whose time interval between them and the target behavior record does not exceed the preset backtracking time as causal pre-sequence records of the target behavior record. The preset preparatory behavior set includes detail browsing behavior, parameter comparison behavior, comment reading behavior, and search behavior. Atomic binding units are used to bind each target behavior record and all its corresponding causal preorder records into an atomic rearrangement unit. The atomic rearrangement unit maintains the relative temporal order of each behavior record in the original behavior sequence. The dual-channel splicing unit is used to arrange all atomic rearrangement units according to the original time order of the target behavior records they contain to form a rearrangement backbone sequence. The behavior records in the original behavior sequence that were not assigned to any atomic rearrangement unit are arranged according to the original time order to form a rearrangement background sequence. The rearrangement backbone sequence is spliced ​​before the rearrangement background sequence to form an intent-focused sequence.

[0010] Furthermore, the causal association identification unit includes: The behavior sequence backtracking subunit is used to extract a segment from the original behavior sequence for each target behavior record. The segment has a time span that does not exceed the preset backtracking duration and the number of behavior records does not exceed the preset backtracking number, and is used as the backtracking window sequence. The three-dimensional correlation calculation subunit is used to calculate the three-dimensional causal correlation between each candidate behavior record and the target behavior record for each candidate behavior record whose behavior type belongs to the preset preparatory behavior set in the backtracking window sequence. The three-dimensional causal correlation is determined by the product of the time decay factor, the behavior transfer probability factor and the information discrimination factor. The time decay factor is calculated based on the time interval between the occurrence of the candidate behavior record and the occurrence of the target behavior record. The larger the time interval, the smaller the time decay factor. The behavior transition probability factor is the probability of the behavior type of a candidate behavior record transitioning to the behavior type of the target behavior record within the historical baseline window; The information distinguishability factor is the information gain value of the behavior type of the candidate behavior record when predicting whether the behavior type of the target behavior record appears. The information gain value is calculated based on the joint probability distribution of the occurrence of the behavior type of the candidate behavior record and the occurrence of the behavior type of the target behavior record in historical behavior data. The causal preorder filtering subunit is used to filter out behavioral records in the backtracking window sequence that have a three-dimensional causal correlation degree exceeding a preset correlation degree threshold, and use them as causal preorder records of the target behavioral record.

[0011] Furthermore, it also includes a contrastive coding module, a semantic difference measurement module, and a gradient modulation module; The contrast encoding module includes a contrast short encoder, which has the same network structure as the short intent encoder. When the intent drift index is higher than the preset intent drift index threshold, the original behavior sequence is input into the contrast short encoder to generate a contrast short intent vector. The semantic difference measurement module is used to calculate the cosine distance between the short-term intent vector and the contrasting short-term intent vector, which is used as the rearrangement gain coefficient. The gradient modulation module is used to use the rearranged gain coefficients as the backpropagation gradient scaling factor of the short-term intent encoder. When updating the parameters of the short-term intent encoder, the larger the backpropagation gradient scaling factor, the larger the parameter update step size.

[0012] Furthermore, the intent drift assessment module also includes: The habit pattern library building unit is used to count the average frequency of each user's behavior type in each fixed time period at fixed intervals, and build a user periodic behavior habit pattern library. The periodic behavior habit pattern library contains the mapping relationship between user identifier, time period identifier, behavior type and average frequency of occurrence. The habit matching unit is used to obtain the time period identifier corresponding to the current observation window, query the user's average occurrence frequency of each behavior type under the current time period identifier from the periodic behavior habit pattern library, and mark the behavior type with an average occurrence frequency exceeding the preset habit frequency threshold as a habit behavior type. The habit stripping unit is used to remove habitual behavior records from the statistics before calculating the difference in behavior type distribution between the current observation window and the historical baseline window.

[0013] Furthermore, the sequence rearrangement module also includes: The rearrangement confidence assessment unit is used to calculate the ratio of the number of target behavior records located at the beginning of the intent-focused sequence to the total number of records in the original behavior sequence, which serves as the rearrangement confidence. The rearrangement confidence indicates the reliability of the sequence rearrangement operation. The sequence length adjustment unit is used to adjust the truncation length of the intent-focused sequence input to the short-term intent encoder according to the rearrangement confidence. The higher the rearrangement confidence, the shorter the truncation length, so that the short-term intent encoder only focuses on the first behavior record with the highest confidence in the intent-focused sequence.

[0014] Furthermore, the fusion output module stores customer profile vectors in a distributed in-memory database and supports real-time point-to-point queries based on user identifiers and conditional searches based on group profile tags.

[0015] This invention provides a key data mining method for customer profiling based on big data, comprising: User behavior data is collected in real time by a behavior sequence acquisition module deployed on user terminal devices, forming a raw behavior sequence sorted by time. The raw behavior sequence is then transmitted to the other modules deployed in the cloud through a cloud collaboration mechanism. The original behavior sequence is divided into multiple observation windows. The difference in behavior type distribution between the current observation window and the historical baseline window is calculated, and an intent drift index is generated based on the difference. When the intent drift index is higher than the preset intent drift index threshold, the behavior records belonging to the preset high intent behavior type in the original behavior sequence are moved forward and clustered in the time dimension to form an intent focus sequence; Input all historical behavior data into the long-term interest encoder to generate long-term interest vectors; The intent-focusing sequence is input into the short-term intent encoder to generate a short-term intent vector; The fusion weights of long-term interest vectors and short-term intention vectors are dynamically generated based on the intention drift index. The higher the intention drift index, the greater the fusion weight of the short-term intention vector. The long-term interest vector and the short-term intent vector are weighted and fused according to the fusion weight to generate a customer profile vector.

[0016] The beneficial effects of this invention compared to existing technologies are as follows: By constructing a closed-loop mechanism of "intent drift assessment, sequence rearrangement, dual-tower encoding, and adaptive fusion," this invention fundamentally solves the problem in existing technologies where, when user intent drifts, evidence of high-intent behaviors in short-term behavior sequences is diluted over time by low-intent noise, leading to a deterioration in the input signal-to-noise ratio of the short-term encoder. By performing conditional rearrangement of the behavior sequence at the data level driven by intent drift, high-intent behavior records are moved forward and aggregated to form an intent-focused sequence. This allows the self-attention mechanism of the short-term encoder to more effectively establish connections between high-value behaviors. Simultaneously, by combining the dynamic control of fusion weights with the intent drift index, collaborative optimization between the data layer and the model layer is achieved, significantly improving the response speed and representation accuracy of customer profiles in scenarios with rapid intent drift.

[0017] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0018] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall architecture diagram of the customer profiling key data mining platform and method based on big data in an embodiment of the present invention. Figure 2 This is a diagram showing the internal structure of the intent drift assessment module in an embodiment of the present invention. Figure 3 This is a diagram showing the internal structure of the sequence rearrangement module in an embodiment of the present invention. Detailed Implementation

[0020] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0021] like Figure 1 , Figure 2 , Figure 3 As shown, this invention provides an embodiment of a customer profiling key data mining platform based on big data, comprising: The behavior sequence acquisition module is used to collect user behavior data in real time and form raw behavior sequences sorted by time. The behavior sequence acquisition module is deployed on the user terminal device and interacts with other modules deployed in the cloud through a cloud collaboration mechanism. The intent drift assessment module is used to divide the original behavior sequence into multiple observation windows, calculate the difference in behavior type distribution between the current observation window and the historical baseline window, and generate an intent drift index based on the difference. The sequence rearrangement module is used to advance and aggregate the behavior records belonging to the preset high-intent behavior type in the original behavior sequence in the time dimension when the intent drift index is higher than the preset intent drift index threshold, so as to form an intent-focused sequence. The dual-tower encoding module includes a long-term interest encoder and a short-term intent encoder. The long-term interest encoder is used to encode the full historical behavior data to generate long-term interest vectors, and the short-term intent encoder is used to encode the intent focus sequence to generate short-term intent vectors. The weight control module is used to dynamically generate the fusion weight of long-term interest vector and short-term intention vector based on the intention drift index. The higher the intention drift index, the greater the fusion weight of the short-term intention vector. The fusion output module is used to weight and fuse long-term interest vectors and short-term intent vectors according to fusion weights to generate customer profile vectors.

[0022] In this embodiment, user behavior data refers to the behavior records generated when a user performs operations on a terminal device. Each behavior record includes a behavior type tag, a behavior object identifier, and a behavior timestamp. The behavior type tags include browsing behavior, clicking behavior, adding to cart behavior, favorites behavior, purchasing behavior, searching behavior, and sharing behavior.

[0023] In this embodiment, the specific method for collecting user behavior data in real time to form a time-sorted original behavior sequence is as follows: the behavior sequence collection module captures the behavior type label, behavior object identifier and behavior timestamp corresponding to each operation performed by the user, and arranges the behavior records in order of the behavior timestamp from smallest to largest to form the original behavior sequence.

[0024] In this embodiment, the behavior sequence acquisition module is deployed on the user terminal device. The specific way in which the behavior sequence acquisition module interacts with the other modules deployed in the cloud through the cloud collaboration mechanism is as follows: after the behavior sequence acquisition module completes the construction of the original behavior sequence locally on the user terminal device, it uploads the original behavior sequence to the cloud server via HTTPS protocol every 2 seconds. The cloud server distributes the original behavior sequence to the intent drift evaluation module, the sequence rearrangement module, and the dual-tower encoding module.

[0025] In this embodiment, the specific way to divide the original behavior sequence into multiple observation windows is as follows: with a window duration of 60 seconds and a sliding step of 30 seconds, the window is slidably truncated starting from the beginning of the original behavior sequence, and each truncated behavior record within the window duration constitutes an observation window.

[0026] In this embodiment, the behavior type distribution refers to the proportion of the number of times each behavior type label appears within an observation window to the total number of times all behavior type labels appear within that observation window.

[0027] In this embodiment, the specific method for calculating the difference in behavior type distribution between the current observation window and the historical baseline window is as follows: Each behavior type is used as a node; the preceding behavior type to the following behavior type in two adjacent behavior records in the original behavior sequence is used as a directed edge; the number of times the same directed edge appears within the observation window is used as the edge weight; this constructs the current transition graph for the current observation window. The historical baseline window is selected as the 5th observation window before the current observation window, constructed in the same way. Then, the weighted in-degree and weighted out-degree of each behavior type node in the current transition graph are calculated, and the weighted in-degree and weighted out-degree are calculated. The in-degree is the sum of the weights of all directed edges pointing to the current node, and the weighted out-degree is the sum of the weights of all directed edges originating from the current node. Then, the rate of change of in-degree for each node is calculated. The rate of change of in-degree is the absolute value of the difference between the weighted in-degree in the current transition graph and the weighted in-degree in the historical transition graph, divided by the weighted in-degree in the historical transition graph. The rate of change of out-degree is calculated similarly. The rate of change of in-degree and the rate of change of out-degree for all nodes are summed with weights, and the weight is the normalized value of the frequency of occurrence of the current node in the current observation window. The summation result is used as the degree of difference in the distribution of behavior types between the current observation window and the historical baseline window.

[0028] In this embodiment, the specific method for generating the intention drift index based on the degree of difference is as follows: the degree of difference of five consecutive observation windows is averaged, and the result of the average is used as the intention drift index.

[0029] In this embodiment, the preset intent drift index threshold is 0.5, and the preset high intent behavior types are purchase behavior and add-to-cart behavior. When the intent drift index is higher than 0.5, it is determined that the user's current intent is in an unstable state, and the sequence reordering module is triggered to perform sequence reordering; when the intent drift index is not higher than 0.5, it is determined that the user's current intent is in a stable state, and sequence reordering is not triggered.

[0030] In this embodiment, the specific method for shifting and aggregating behavior records belonging to the preset high-intent behavior type in the original behavior sequence to form an intent-focused sequence in the time dimension is as follows: the behavior records in the original behavior sequence with the behavior type label of purchase behavior or add-to-cart behavior are extracted to the beginning of the sequence in the original time order, and the remaining behavior records are arranged to the end of the sequence in the original time order, and spliced ​​together to form an intent-focused sequence.

[0031] In this embodiment, the full historical behavior data refers to all the behavioral records generated by the user on the platform from the date of registration to the present moment.

[0032] In this embodiment, the long-term interest encoder is used to encode the full historical behavior data to generate long-term interest vectors in the following way: The long-term interest encoder adopts a deep Transformer encoder with a structure of 6 stacked Transformer encoding layers, each layer containing 8 self-attention sub-layers and feedforward neural network sub-layers, and the hidden layer dimension is 256; the input is the daily behavior embedding vector sequence obtained by aggregating the full historical behavior data by day, and the daily behavior embedding vector is constructed by summing and averaging the pre-trained word vectors corresponding to the behavior type labels of all behavior records within that day; the output is a 128-dimensional long-term interest vector obtained by linearly transforming the hidden state at the end of the sequence after encoding the daily behavior embedding vector sequence.

[0033] In this embodiment, the short-term intent encoder is used to encode the intent focus sequence to generate a short-term intent vector in the following way: the short-term intent encoder adopts a shallow Transformer encoder with a structure of two stacked Transformer encoding layers, each layer containing a 4-head self-attention sub-layer and a feedforward neural network sub-layer, and the hidden layer dimension is 128; the input is the behavior embedding sequence formed by the pre-trained word vectors corresponding to the behavior type labels of each behavior record in the intent focus sequence arranged in order; the output is a 128-dimensional short-term intent vector obtained by linear transformation of the hidden state at the end of the sequence after encoding the behavior embedding sequence.

[0034] In this embodiment, the specific method of dynamically generating fusion weights based on the intent drift index and showing a positive correlation is as follows: the fusion weight is the proportional coefficient of the short-term intent vector in the weighted fusion, with a value of 0 to 1. The fusion weight of the long-term interest vector is 1 minus the fusion weight of the short-term intent vector. The higher the intent drift index, the greater the fusion weight of the short-term intent vector.

[0035] In this embodiment, the specific method for generating a customer profile vector by weighted fusion of long-term interest vector and short-term intent vector according to fusion weight is as follows: multiply the long-term interest vector by the fusion weight of the long-term interest vector, multiply the short-term intent vector by the fusion weight of the short-term intent vector, and add the two weighted results element by element to obtain the customer profile vector.

[0036] Furthermore, the intent drift assessment module includes: The transition graph construction unit is used to construct the current transition graph of the current observation window and the historical transition graph of the historical benchmark window, respectively, with behavior type as the node, the previous behavior type pointing to the next behavior type of two consecutive behavior records in the original behavior sequence as the directed edge, and the occurrence number of consecutive behavior pairs as the weight of the directed edge. The in-degree decomposition unit is used to calculate the weighted in-degree and weighted out-degree of each behavior type node in the current transition graph. The weighted in-degree is the sum of the weights of all directed edges pointing to the current behavior type node, and the weighted out-degree is the sum of the weights of all directed edges originating from the current behavior type node. The transition bias calculation unit is used to calculate the rate of change of the weighted in-degree of each behavior type node in the current transition graph and the weighted in-degree of the historical transition graph, as well as the rate of change of the weighted out-degree of the current transition graph and the weighted out-degree of the historical transition graph. The difference calculation unit is used to perform a weighted summation of the in-degree change rate and out-degree change rate of all behavior type nodes. The weight is the normalized value of the frequency of occurrence of behavior type nodes in the current observation window. The weighted summation result is used as the difference in behavior type distribution between the current observation window and the historical baseline window. The index generation unit is used to perform a moving average of the differences across multiple consecutive observation windows to obtain the intention drift index.

[0037] In this embodiment, the historical reference window is an observation window selected before the current observation window. The selection method is to take the fifth observation window before the current observation window. If there are fewer than five observation windows before the current observation window, the observation window immediately preceding the current observation window is taken as the historical reference window.

[0038] In this embodiment, the specific method for constructing the current transition graph of the current observation window and the historical transition graph of the historical benchmark window using behavior type as nodes, the direction from the previous behavior type to the next behavior type of two consecutive behavior records in the original behavior sequence as directed edges, and the occurrence frequency of consecutive behavior pairs as the weight of the directed edges is as follows: traverse all two consecutive behavior records in the observation window, take the behavior type label of the previous behavior record as the starting node, and the behavior type label of the next behavior record as the ending node, and construct a directed edge from the starting node to the ending node. If the directed edge already exists, increment its weight by 1; if it does not exist, create the directed edge and set its weight to 1. Perform the above traversal to construct the current transition graph for the current observation window, and perform the above traversal to construct the historical transition graph for the historical benchmark window.

[0039] In this embodiment, the specific method for calculating the rate of change of in-degree between the weighted in-degree of each behavior type node in the current transition graph and the weighted in-degree in the historical transition graph is as follows: The weighted in-degree is equal to the sum of the weights of all directed edges pointing to the current behavior type node. The difference between the weighted in-degree of the current node in the current transition graph and the weighted in-degree of the current node in the historical transition graph is divided by the weighted in-degree of the current node in the historical transition graph, and the resulting ratio is used as the rate of change of in-degree. The rate of change of out-degree is calculated similarly. The difference between the weighted out-degree of the current node in the current transition graph and the weighted out-degree of the current node in the historical transition graph is divided by the weighted out-degree of the current node in the historical transition graph, and the resulting ratio is used as the rate of change of out-degree. If the denominator is zero, the rate of change is set to 0.

[0040] In this embodiment, the specific method for weighted summation of the in-degree and out-degree change rates of all behavior type nodes, with the weight being the normalized value of the frequency of occurrence of the behavior type node in the current observation window, is as follows: the normalized value is equal to the number of times the current behavior type node appears in the current observation window divided by the total number of times all behavior type nodes appear in the current observation window; the in-degree and out-degree change rates of each behavior type node are first summed, then multiplied by the normalized value of the current node, and finally the product results of all nodes are accumulated, and the accumulated result is used as the difference degree of behavior type distribution between the current observation window and the historical baseline window.

[0041] In this embodiment, the specific method for obtaining the intention drift index by moving average of the differences of multiple consecutive observation windows is as follows: take the differences of the current observation window and the four consecutive observation windows before it, for a total of five observation windows, calculate the arithmetic mean of these five differences, and use it as the intention drift index corresponding to the current observation window.

[0042] Furthermore, the transfer bias calculation unit is also used for: Obtain the historical weighted out-degree and historical weighted in-degree of each behavior type node in the historical baseline window; For each behavior type node in the current transition graph, when the historical weighted out-degree is greater than zero, the ratio of the node's weighted out-degree in the current transition graph to the historical weighted out-degree is used as the out-degree anomaly coefficient; when the historical weighted out-degree is zero, the out-degree anomaly coefficient is set to a preset upper limit value. When the historical weighted in-degree is greater than zero, the ratio of the node's weighted in-degree in the current transition graph to the historical weighted in-degree is used as the in-degree anomaly coefficient; when the historical weighted in-degree is zero, the in-degree anomaly coefficient is set to a preset upper limit value. The product of the out-degree anomaly coefficient and the in-degree anomaly coefficient is used as the transition mode anomaly coefficient of the node. The difference calculation unit is also used to perform a weighted summation of the transition mode anomaly coefficients of all behavior type nodes, using the transition mode anomaly coefficient as the basis for weighted summation. The weight is the normalized value of the frequency of occurrence of behavior type nodes in the current observation window, and the weighted summation result is used as the difference in behavior type distribution between the current observation window and the historical baseline window.

[0043] In this embodiment, the specific method for obtaining the historical weighted out-degree and historical weighted in-degree of each behavior type node in the historical baseline window is as follows: after constructing the historical transition graph of the historical baseline window, for each behavior type node in the historical transition graph, calculate the sum of the weights of all directed edges pointing to the current node as the historical weighted in-degree, and calculate the sum of the weights of all directed edges originating from the current node as the historical weighted out-degree.

[0044] In this embodiment, the preset upper limit value is 3.0. When the historical weighted out-degree is zero, the out-degree anomaly coefficient is set to 3.0, and when the historical weighted in-degree is zero, the in-degree anomaly coefficient is set to 3.0.

[0045] In this embodiment, the specific method of using the product of the out-degree anomaly coefficient and the in-degree anomaly coefficient as the transition mode anomaly coefficient of the node is as follows: for each behavior type node, the calculated out-degree anomaly coefficient is multiplied by the in-degree anomaly coefficient, and the product result is used as the transition mode anomaly coefficient of the current node. The larger the transition mode anomaly coefficient, the greater the degree to which the current node deviates from the historical benchmark in the transition structure.

[0046] In this embodiment, the specific method of using the transition pattern anomaly coefficient as the basis for weighted summation, weighting and summing the transition pattern anomaly coefficients of all behavior type nodes, and using the normalized value of the frequency of occurrence of behavior type nodes in the current observation window as the weight is as follows: the transition pattern anomaly coefficient is used to replace the in-degree change rate and out-degree change rate as the summed value, the transition pattern anomaly coefficient of each behavior type node is multiplied by the normalized value of the frequency of occurrence of the current node in the current observation window, the normalized value is equal to the number of occurrences of the current node divided by the total number of occurrences of all nodes in the current observation window, the product results of all nodes are summed, and the summed result is used as the difference degree of behavior type distribution between the current observation window and the historical baseline window.

[0047] Furthermore, the sequence rearrangement module includes: The causal association identification unit is used to trace back in the original behavior sequence for each target behavior record that belongs to the preset high-intent behavior type. It identifies behavior records that occurred before the target behavior record, belong to the preset preparatory behavior set, and whose time interval between them and the target behavior record does not exceed the preset backtracking time as causal pre-sequence records of the target behavior record. The preset preparatory behavior set includes detail browsing behavior, parameter comparison behavior, comment reading behavior, and search behavior. Atomic binding units are used to bind each target behavior record and all its corresponding causal preorder records into an atomic rearrangement unit. The atomic rearrangement unit maintains the relative temporal order of each behavior record in the original behavior sequence. The dual-channel splicing unit is used to arrange all atomic rearrangement units according to the original time order of the target behavior records they contain to form a rearrangement backbone sequence. The behavior records in the original behavior sequence that were not assigned to any atomic rearrangement unit are arranged according to the original time order to form a rearrangement background sequence. The rearrangement backbone sequence is spliced ​​before the rearrangement background sequence to form an intent-focused sequence.

[0048] In this embodiment, the preset high-intent behavior types are purchase behavior and add-to-cart behavior. The preset set of pre-set behaviors includes detail browsing behavior, parameter comparison behavior, review reading behavior, and search behavior. The preset backtracking time is 120 seconds.

[0049] In this embodiment, the specific method for tracing back each target behavior record belonging to the preset high-intent behavior type in the original behavior sequence is as follows: for each target behavior record with the behavior type tag of purchase behavior or add-to-cart behavior, trace back 120 seconds from the behavior timestamp of the target behavior record, filter out behavior records in the original behavior sequence whose behavior timestamp falls within this time range and whose behavior type tag belongs to detail browsing behavior, parameter comparison behavior, comment reading behavior, or search behavior, and identify these behavior records as the causal preceding records of the target behavior record.

[0050] In this embodiment, the behavior of browsing details refers to the behavior of a user staying on the product details page for more than 10 seconds, the behavior of comparing parameters refers to the behavior of a user opening the product parameter comparison interface, the behavior of reading comments refers to the behavior of a user browsing the product comment section, and the behavior of searching refers to the behavior of a user entering keywords in the search box and submitting the search.

[0051] In this embodiment, the specific way to bind each target behavior record and all corresponding causal prequel records into an atomic rearrangement unit is as follows: a target behavior record and all its identified causal prequel records are treated as an indivisible processing unit. Within this processing unit, each behavior record maintains its relative position according to its original time order in the original behavior sequence.

[0052] In this embodiment, the specific method for arranging all atomic rearrangement units according to the original chronological order of the target behavior records they contain to form the rearrangement backbone sequence is as follows: The behavior timestamps of the target behavior records contained in different atomic rearrangement units are compared, and all atomic rearrangement units are arranged sequentially in ascending order of behavior timestamps to form the rearrangement backbone sequence. Behavior records in the original behavior sequence that are not assigned to any atomic rearrangement unit are arranged in their original chronological order to form the rearrangement background sequence. The rearrangement backbone sequence is then concatenated before the rearrangement background sequence, and the resulting complete sequence serves as the intent-focusing sequence.

[0053] Furthermore, the causal association identification unit includes: The behavior sequence backtracking subunit is used to extract a segment from the original behavior sequence for each target behavior record. The segment has a time span that does not exceed the preset backtracking duration and the number of behavior records does not exceed the preset backtracking number, and is used as the backtracking window sequence. The three-dimensional correlation calculation subunit is used to calculate the three-dimensional causal correlation between each candidate behavior record and the target behavior record for each candidate behavior record whose behavior type belongs to the preset preparatory behavior set in the backtracking window sequence. The three-dimensional causal correlation is determined by the product of the time decay factor, the behavior transfer probability factor and the information discrimination factor. The time decay factor is calculated based on the time interval between the occurrence of the candidate behavior record and the occurrence of the target behavior record. The larger the time interval, the smaller the time decay factor. The behavior transition probability factor is the probability of the behavior type of a candidate behavior record transitioning to the behavior type of the target behavior record within the historical baseline window; The information distinguishability factor is the information gain value of the behavior type of the candidate behavior record when predicting whether the behavior type of the target behavior record appears. The information gain value is calculated based on the joint probability distribution of the occurrence of the behavior type of the candidate behavior record and the occurrence of the behavior type of the target behavior record in historical behavior data. The causal preorder filtering subunit is used to filter out behavioral records in the backtracking window sequence that have a three-dimensional causal correlation degree exceeding a preset correlation degree threshold, and use them as causal preorder records of the target behavioral record.

[0054] In this embodiment, the time span refers to the absolute value of the difference between the behavior timestamp of the candidate behavior record and the behavior timestamp of the target behavior record, in seconds.

[0055] In this embodiment, the number of behavior records refers to the number of behavior records contained in the backtracking window sequence. The preset number of backtracking records is 50, and the preset backtracking duration is 120 seconds.

[0056] In this embodiment, the specific method for extracting a segment from the original behavior sequence for each target behavior record with a time span not exceeding a preset backtracking duration and a number of behavior records not exceeding a preset backtracking number is as follows: Starting from the target behavior record, backtracking is performed line by line in the original behavior sequence. Behavior records whose time span between the behavior timestamp and the behavior timestamp of the target behavior record does not exceed 120 seconds are included in the backtracking window sequence. At the same time, the total number of included behavior records does not exceed 50. Backtracking stops when either the time span or the number of behavior records reaches the upper limit. The included behavior records are arranged in the original time order to form the backtracking window sequence.

[0057] In this embodiment, the specific method for calculating the three-dimensional causal correlation degree for each candidate behavior record whose behavior type belongs to the preset preparatory behavior set in the backtracking window sequence is as follows: the three-dimensional causal correlation degree is equal to the product of the time decay factor, the behavior transfer probability factor, and the information discrimination factor.

[0058] In this embodiment, the time decay factor is calculated as follows: the time decay factor is equal to the result of a power operation with the natural constant e as the base and the negative time interval divided by 120 as the exponent. The time interval is the absolute value of the difference between the behavior timestamp of the candidate behavior record and the behavior timestamp of the target behavior record, and the unit is seconds.

[0059] In this embodiment, the behavior transfer probability factor is calculated as follows: after the behavior type label of the candidate behavior record appears in the historical benchmark window, the number of times the behavior type label of the target behavior record appears within 120 seconds is counted, and the result is divided by the total number of times the behavior type label of the candidate behavior record appears in the historical benchmark window. The ratio obtained is used as the behavior transfer probability factor.

[0060] In this embodiment, the specific calculation method of the information distinguishability factor is as follows: whether the behavior type label of the candidate behavior record appears in the historical behavior data is used as a feature, and whether the behavior type label of the target behavior record appears is used as a label. The information gain value of the behavior type label of the candidate behavior record to the behavior type label of the target behavior record is calculated. The information gain value is calculated based on the joint probability distribution of the appearance of the behavior type label of the candidate behavior record and the appearance of the behavior type label of the target behavior record. The four combinations in the joint probability distribution are candidate and target, candidate and target not appear, candidate and target not appear, and candidate and target not appear. The probability of each combination is equal to the number of times the combination appears in the historical behavior data divided by the total number of records in the historical behavior data.

[0061] In this embodiment, the preset correlation threshold is 0.3. Behavioral records with a three-dimensional causal correlation greater than 0.3 are selected from the backtracking window sequence as the causal preceding records of the target behavioral record.

[0062] In this embodiment, the specific method for selecting behavioral records with a three-dimensional causal correlation degree exceeding a preset correlation degree threshold as causal precedent records of the target behavioral record in the backtracking window sequence is as follows: for each candidate behavioral record in the backtracking window sequence, the calculated three-dimensional causal correlation degree is compared with the preset correlation degree threshold of 0.3. Candidate behavioral records with a three-dimensional causal correlation degree greater than 0.3 are marked as causal precedent records of the target behavioral record, while candidate behavioral records with a three-dimensional causal correlation degree less than or equal to 0.3 are not marked.

[0063] Furthermore, it also includes a contrastive coding module, a semantic difference measurement module, and a gradient modulation module; The contrast encoding module includes a contrast short encoder, which has the same network structure as the short intent encoder. When the intent drift index is higher than the preset intent drift index threshold, the original behavior sequence is input into the contrast short encoder to generate a contrast short intent vector. The semantic difference measurement module is used to calculate the cosine distance between the short-term intent vector and the contrasting short-term intent vector, which is used as the rearrangement gain coefficient. The gradient modulation module is used to use the rearranged gain coefficients as the backpropagation gradient scaling factor of the short-term intent encoder. When updating the parameters of the short-term intent encoder, the larger the backpropagation gradient scaling factor, the larger the parameter update step size.

[0064] In this embodiment, the comparison short encoder and the short intent encoder have the same network structure as follows: the comparison short encoder also adopts a shallow Transformer encoder, with a structure of two stacked Transformer encoding layers. Each layer contains four self-attention sublayers and a feedforward neural network sublayer, and the hidden layer dimension is 128, which is exactly the same as the number of network layers, the number of heads, and the hidden layer dimension of the short intent encoder. When the intent drift index is higher than the preset intent drift index threshold of 0.5, the behavior embedding sequence formed by arranging the pre-trained word vectors corresponding to the behavior type labels of each behavior record in the original behavior sequence in order is input into the comparison short encoder. After encoding the behavior embedding sequence, the comparison short encoder takes the hidden state at the end of the sequence and outputs a 128-dimensional comparison short intent vector through linear transformation.

[0065] In this embodiment, the specific method for calculating the cosine distance between the short-term intent vector and the comparative short-term intent vector as the rearrangement gain coefficient is as follows: multiply the short-term intent vector and the comparative short-term intent vector element by element and sum them, then divide by the product of the magnitude of the short-term intent vector and the magnitude of the comparative short-term intent vector. Subtract 1 from the resulting cosine similarity value to obtain the cosine distance, which is used as the rearrangement gain coefficient.

[0066] In this embodiment, the backpropagation gradient scaling factor of the short-term intent encoder is a coefficient multiplied by the gradient of the loss function during backpropagation training of the short-term intent encoder, used to adjust the step size of each parameter update.

[0067] In this embodiment, the specific method of using the rearranged gain coefficient as the backpropagation gradient scaling factor of the short-term intent encoder is as follows: During the backpropagation training of the short-term intent encoder, the original gradient values ​​calculated by the loss function with respect to the parameters of each layer of the short-term intent encoder are multiplied by the rearranged gain coefficient to obtain the scaled gradient values. The scaled gradient values ​​are used to update the parameters of the short-term intent encoder; the larger the rearranged gain coefficient, the larger the scaled gradient value, and the larger the parameter update step size. During the model training phase, taking the user's next line prediction task as an example, the prediction loss is calculated based on the customer profile vector. The gradient modulation module is used to multiply the rearranged gain coefficient and the gradient of the prediction loss with respect to the parameters of the short-term intent encoder when updating the parameters of the short-term intent encoder through backpropagation to obtain the scaled gradient, and then update the parameters of the short-term intent encoder with the scaled gradient.

[0068] Furthermore, the intent drift assessment module also includes: The habit pattern library building unit is used to count the average frequency of each user's behavior type in each fixed time period at fixed intervals, and build a user periodic behavior habit pattern library. The periodic behavior habit pattern library contains the mapping relationship between user identifier, time period identifier, behavior type and average frequency of occurrence. The habit matching unit is used to obtain the time period identifier corresponding to the current observation window, query the user's average occurrence frequency of each behavior type under the current time period identifier from the periodic behavior habit pattern library, and mark the behavior type with an average occurrence frequency exceeding the preset habit frequency threshold as a habit behavior type. The habit stripping unit is used to remove habitual behavior records from the statistics before calculating the difference in behavior type distribution between the current observation window and the historical baseline window.

[0069] In this embodiment, the fixed cycle is 7 days, and the fixed time period is to divide a day of 24 hours into 1 hour into a time period, for a total of 24 time periods. Each time period is identified by the hour.

[0070] In this embodiment, the specific method for calculating the average frequency of each user's behavior type within each fixed time period at a fixed cycle is as follows: Calculations are performed every 7 days. For each user, the number of times each behavior type tag appears in each 1-hour time period within the past 7 days is counted, and this number is divided by 7 to obtain the average frequency of that behavior type for that user within that time period. The periodic behavior habit pattern library contains a mapping relationship between user identifier, time period identifier, behavior type, and average frequency. Using the user identifier and time period identifier as a joint query key, a list of the average frequency of each behavior type for that user within that time period is returned.

[0071] In this embodiment, the preset habit frequency threshold is 3.0. The specific method for obtaining the time period identifier corresponding to the current observation window is as follows: take the hour when the start timestamp of the current observation window is located as the time period identifier, use the user identifier and the time period identifier as the joint query key, query the average occurrence frequency of each behavior type of the user in the periodic behavior habit pattern library, and mark the behavior type with an average occurrence frequency of more than 3.0 as the habitual behavior type.

[0072] In this embodiment, the specific method for removing behavior records belonging to habitual behavior types from the statistics before calculating the difference in behavior type distribution between the current observation window and the historical baseline window is as follows: traverse all behavior records in the current observation window and the historical baseline window, remove behavior records whose behavior type labels belong to the labeled habitual behavior types from the behavior record set of their respective windows, and use the remaining behavior records of the current observation window and the remaining behavior records of the historical baseline window for subsequent transition graph construction and difference calculation, respectively.

[0073] Furthermore, the sequence rearrangement module also includes: The rearrangement confidence assessment unit is used to calculate the ratio of the number of target behavior records located at the beginning of the intent-focused sequence to the total number of records in the original behavior sequence, which serves as the rearrangement confidence. The rearrangement confidence indicates the reliability of the sequence rearrangement operation. The sequence length adjustment unit is used to adjust the truncation length of the intent-focused sequence input to the short-term intent encoder according to the rearrangement confidence. The higher the rearrangement confidence, the shorter the truncation length, so that the short-term intent encoder only focuses on the first behavior record with the highest confidence in the intent-focused sequence.

[0074] In this embodiment, the number of target behavior records at the beginning of the intent-focused sequence refers to the total number of target behavior records in the intent-focused sequence from the first behavior record to the end of all target behavior records. The target behavior records are behavior records whose behavior type tags belong to purchase behavior or add-to-cart behavior.

[0075] In this embodiment, the specific method for adjusting the truncation length of the intent-focused sequence input to the short-term intent encoder based on the rearrangement confidence is as follows: the total length of the intent-focused sequence is multiplied by 1 and the difference in rearrangement confidence is subtracted to obtain the truncation length. The higher the rearrangement confidence, the shorter the truncation length, and the fewer the preceding behavior records are retained, so that the short-term intent encoder only focuses on the preceding behavior records with the highest confidence in the intent-focused sequence. When the rearrangement confidence is 1.0, the truncation length is the shortest, and only a small number of behavior records at the very beginning of the intent-focused sequence are retained. When the rearrangement confidence is 0.2, the truncation length is longer, and most of the behavior records in the intent-focused sequence are retained to compensate for the context information that may be lost when the rearrangement quality is insufficient.

[0076] Furthermore, the fusion output module stores customer profile vectors in a distributed in-memory database and supports real-time point-to-point queries based on user identifiers and conditional searches based on group profile tags.

[0077] In this embodiment, the distributed memory database uses a Redis cluster, which consists of multiple Redis instances. Each instance is responsible for storing a portion of customer profile vector data, and the instances are sharded using a consistent hashing algorithm.

[0078] In this embodiment, the user identifier is a unique identification string assigned to the user by the platform, consisting of numbers and letters, used to distinguish different users.

[0079] In this embodiment, the specific method of real-time point query based on user identifier is as follows: the external business system uses the user identifier as the query key and initiates a query request to the distributed memory database through the Redis GET command. The distributed memory database locates the Redis instance storing the user data according to the consistent hashing algorithm and returns the customer profile vector corresponding to the user.

[0080] In this embodiment, the group profile label is a classification label defined for each user group after clustering analysis of the customer profile vector. The group profile labels include high-spending intention group, price-sensitive group, new product preference group, and brand-loyal group.

[0081] In this embodiment, the specific method of conditional retrieval based on group profile tags is as follows: the external business system uses group profile tags as query conditions, traverses all customer profile vectors in the distributed in-memory database through the Redis SCAN command, filters out customer profile vectors that match the group profile tags, and returns them.

[0082] This invention provides an embodiment of a key data mining method for customer profiling based on big data, comprising: User behavior data is collected in real time by a behavior sequence acquisition module deployed on user terminal devices, forming a raw behavior sequence sorted by time. The raw behavior sequence is then transmitted to the other modules deployed in the cloud through a cloud collaboration mechanism. The original behavior sequence is divided into multiple observation windows. The difference in behavior type distribution between the current observation window and the historical baseline window is calculated, and an intent drift index is generated based on the difference. When the intent drift index is higher than the preset intent drift index threshold, the behavior records belonging to the preset high intent behavior type in the original behavior sequence are moved forward and clustered in the time dimension to form an intent focus sequence; Input all historical behavior data into the long-term interest encoder to generate long-term interest vectors; The intent-focusing sequence is input into the short-term intent encoder to generate a short-term intent vector; The fusion weights of long-term interest vectors and short-term intention vectors are dynamically generated based on the intention drift index. The higher the intention drift index, the greater the fusion weight of the short-term intention vector. The long-term interest vector and the short-term intent vector are weighted and fused according to the fusion weight to generate a customer profile vector.

[0083] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A customer profiling key data mining platform based on big data, characterized in that, include: The behavior sequence acquisition module is used to collect user behavior data in real time and form raw behavior sequences sorted by time. The behavior sequence acquisition module is deployed on the user terminal device and interacts with other modules deployed in the cloud through a cloud collaboration mechanism. The intent drift assessment module is used to divide the original behavior sequence into multiple observation windows, calculate the difference in behavior type distribution between the current observation window and the historical baseline window, and generate an intent drift index based on the difference. The sequence rearrangement module is used to advance and aggregate the behavior records belonging to the preset high-intent behavior type in the original behavior sequence in the time dimension when the intent drift index is higher than the preset intent drift index threshold, so as to form an intent-focused sequence. The dual-tower encoding module includes a long-term interest encoder and a short-term intent encoder. The long-term interest encoder is used to encode the full historical behavior data to generate long-term interest vectors, and the short-term intent encoder is used to encode the intent focus sequence to generate short-term intent vectors. The weight control module is used to dynamically generate the fusion weight of long-term interest vector and short-term intention vector based on the intention drift index. The higher the intention drift index, the greater the fusion weight of the short-term intention vector. The fusion output module is used to weight and fuse long-term interest vectors and short-term intent vectors according to fusion weights to generate customer profile vectors.

2. The customer profiling key data mining platform based on big data according to claim 1, characterized in that, The intent drift assessment module includes: The transition graph construction unit is used to construct the current transition graph of the current observation window and the historical transition graph of the historical benchmark window, respectively, with behavior type as the node, the previous behavior type pointing to the next behavior type of two consecutive behavior records in the original behavior sequence as the directed edge, and the occurrence number of consecutive behavior pairs as the weight of the directed edge. The in-degree decomposition unit is used to calculate the weighted in-degree and weighted out-degree of each behavior type node in the current transition graph. The weighted in-degree is the sum of the weights of all directed edges pointing to the current behavior type node, and the weighted out-degree is the sum of the weights of all directed edges originating from the current behavior type node. The transition bias calculation unit is used to calculate the rate of change of the weighted in-degree of each behavior type node in the current transition graph and the weighted in-degree of the historical transition graph, as well as the rate of change of the weighted out-degree of the current transition graph and the weighted out-degree of the historical transition graph. The difference calculation unit is used to perform a weighted summation of the in-degree change rate and out-degree change rate of all behavior type nodes. The weight is the normalized value of the frequency of occurrence of behavior type nodes in the current observation window. The weighted summation result is used as the difference in behavior type distribution between the current observation window and the historical baseline window. The index generation unit is used to perform a moving average of the differences across multiple consecutive observation windows to obtain the intention drift index.

3. The customer profiling key data mining platform based on big data according to claim 2, characterized in that, The transfer bias calculation unit is also used for: Obtain the historical weighted out-degree and historical weighted in-degree of each behavior type node in the historical baseline window; For each behavior type node in the current transition graph, when the historical weighted out-degree is greater than zero, the ratio of the node's weighted out-degree in the current transition graph to the historical weighted out-degree is used as the out-degree anomaly coefficient; when the historical weighted out-degree is zero, the out-degree anomaly coefficient is set to a preset upper limit value. When the historical weighted in-degree is greater than zero, the ratio of the node's weighted in-degree in the current transition graph to the historical weighted in-degree is used as the in-degree anomaly coefficient; when the historical weighted in-degree is zero, the in-degree anomaly coefficient is set to a preset upper limit value. The product of the out-degree anomaly coefficient and the in-degree anomaly coefficient is used as the transition mode anomaly coefficient of the node. The difference calculation unit is also used to perform a weighted summation of the transition mode anomaly coefficients of all behavior type nodes, using the transition mode anomaly coefficient as the basis for weighted summation. The weight is the normalized value of the frequency of occurrence of behavior type nodes in the current observation window, and the weighted summation result is used as the difference in behavior type distribution between the current observation window and the historical baseline window.

4. The customer profiling key data mining platform based on big data according to claim 1, characterized in that, The sequence rearrangement module includes: The causal association identification unit is used to trace back in the original behavior sequence for each target behavior record that belongs to the preset high-intent behavior type. It identifies behavior records that occurred before the target behavior record, belong to the preset preparatory behavior set, and whose time interval between them and the target behavior record does not exceed the preset backtracking time as causal pre-sequence records of the target behavior record. The preset preparatory behavior set includes detail browsing behavior, parameter comparison behavior, comment reading behavior, and search behavior. Atomic binding units are used to bind each target behavior record and all its corresponding causal preorder records into an atomic rearrangement unit. The atomic rearrangement unit maintains the relative temporal order of each behavior record in the original behavior sequence. The dual-channel splicing unit is used to arrange all atomic rearrangement units according to the original time order of the target behavior records they contain to form a rearrangement backbone sequence. The behavior records in the original behavior sequence that were not assigned to any atomic rearrangement unit are arranged according to the original time order to form a rearrangement background sequence. The rearrangement backbone sequence is spliced ​​before the rearrangement background sequence to form an intent-focused sequence.

5. The customer profiling key data mining platform based on big data according to claim 4, characterized in that, The causal association identification unit includes: The behavior sequence backtracking subunit is used to extract a segment from the original behavior sequence for each target behavior record. The segment has a time span that does not exceed the preset backtracking duration and the number of behavior records does not exceed the preset backtracking number, and is used as the backtracking window sequence. The three-dimensional correlation calculation subunit is used to calculate the three-dimensional causal correlation between each candidate behavior record and the target behavior record for each candidate behavior record whose behavior type belongs to the preset preparatory behavior set in the backtracking window sequence. The three-dimensional causal correlation is determined by the product of the time decay factor, the behavior transfer probability factor and the information discrimination factor. The time decay factor is calculated based on the time interval between the occurrence of the candidate behavior record and the occurrence of the target behavior record. The larger the time interval, the smaller the time decay factor. The behavior transition probability factor is the probability of the behavior type of a candidate behavior record transitioning to the behavior type of the target behavior record within the historical baseline window; The information distinguishability factor is the information gain value of the behavior type of the candidate behavior record when predicting whether the behavior type of the target behavior record appears. The information gain value is calculated based on the joint probability distribution of the occurrence of the behavior type of the candidate behavior record and the occurrence of the behavior type of the target behavior record in historical behavior data. The causal preorder filtering subunit is used to filter out behavioral records in the backtracking window sequence that have a three-dimensional causal correlation degree exceeding a preset correlation degree threshold, and use them as causal preorder records of the target behavioral record.

6. The customer profiling key data mining platform based on big data according to claim 1, characterized in that, It also includes a contrast coding module, a semantic difference measurement module, and a gradient modulation module; The contrast encoding module includes a contrast short encoder, which has the same network structure as the short intent encoder. When the intent drift index is higher than the preset intent drift index threshold, the original behavior sequence is input into the contrast short encoder to generate a contrast short intent vector. The semantic difference measurement module is used to calculate the cosine distance between the short-term intent vector and the contrasting short-term intent vector, which is used as the rearrangement gain coefficient. The gradient modulation module is used to use the rearranged gain coefficients as the backpropagation gradient scaling factor of the short-term intent encoder. When updating the parameters of the short-term intent encoder, the larger the backpropagation gradient scaling factor, the larger the parameter update step size.

7. The customer profiling key data mining platform based on big data according to claim 1, characterized in that, The intent drift assessment module also includes: The habit pattern library building unit is used to count the average frequency of each user's behavior type in each fixed time period at fixed intervals, and build a user periodic behavior habit pattern library. The periodic behavior habit pattern library contains the mapping relationship between user identifier, time period identifier, behavior type and average frequency of occurrence. The habit matching unit is used to obtain the time period identifier corresponding to the current observation window, query the user's average occurrence frequency of each behavior type under the current time period identifier from the periodic behavior habit pattern library, and mark the behavior type with an average occurrence frequency exceeding the preset habit frequency threshold as a habit behavior type. The habit stripping unit is used to remove habitual behavior records from the statistics before calculating the difference in behavior type distribution between the current observation window and the historical baseline window.

8. The customer profiling key data mining platform based on big data according to claim 1, characterized in that, The sequence rearrangement module also includes: The rearrangement confidence assessment unit is used to calculate the ratio of the number of target behavior records located at the beginning of the intent-focused sequence to the total number of records in the original behavior sequence, which serves as the rearrangement confidence. The rearrangement confidence indicates the reliability of the sequence rearrangement operation. The sequence length adjustment unit is used to adjust the truncation length of the intent-focused sequence input to the short-term intent encoder according to the rearrangement confidence. The higher the rearrangement confidence, the shorter the truncation length, so that the short-term intent encoder only focuses on the first behavior record with the highest confidence in the intent-focused sequence.

9. The customer profiling key data mining platform based on big data according to any one of claims 1 to 8, characterized in that, The fusion output module stores customer profile vectors in a distributed in-memory database and supports real-time point-to-point queries based on user identifiers and conditional searches based on group profile tags.

10. A method for key data mining of customer profiling based on big data, characterized in that, include: User behavior data is collected in real time by a behavior sequence acquisition module deployed on user terminal devices, forming a raw behavior sequence sorted by time. The raw behavior sequence is then transmitted to the other modules deployed in the cloud through a cloud collaboration mechanism. The original behavior sequence is divided into multiple observation windows. The difference in behavior type distribution between the current observation window and the historical baseline window is calculated, and an intent drift index is generated based on the difference. When the intent drift index is higher than the preset intent drift index threshold, the behavior records belonging to the preset high intent behavior type in the original behavior sequence are moved forward and clustered in the time dimension to form an intent focus sequence; Input all historical behavior data into the long-term interest encoder to generate long-term interest vectors; The intent-focusing sequence is input into the short-term intent encoder to generate a short-term intent vector; The fusion weights of long-term interest vectors and short-term intention vectors are dynamically generated based on the intention drift index. The higher the intention drift index, the greater the fusion weight of the short-term intention vector. The long-term interest vector and the short-term intent vector are weighted and fused according to the fusion weight to generate a customer profile vector.

Images