Customer grouping method and system fusing multi-modal RFM analysis and deep learning

Abstract

The invention discloses a multi-modal RFM analysis and deep learning fused customer grouping method and system, and the method comprises the following steps: collecting voice, image and structured data through multiple channels, carrying out the quartile 3-score standardization processing of an original RFM index, converting the original RFM index into 0-3 scores, and screening effective data; extracting each modal feature by using a deep learning model, obtaining an emotional state of voice through a CRNN, extracting visual attributes of an image through a pre-trained CNN, extracting high-order behavior features of structured data through a full-connection neural network, splicing to form a high-dimensional multi-modal fusion vector, and constructing an omnibearing customer portrait; before K-means clustering, a business-oriented and stability-driven multi-index fusion K value determination framework is adopted, internal, theoretical and stability indexes are comprehensively evaluated, an optimal K value is selected through normalized weighted fusion, accurate grouping is achieved, and the problem that a traditional single data source portrait is one-sided is solved.

Description

Technical Field

[0001] This application relates to the field of customer analytics technology, and in particular to a customer segmentation method and system that integrates multimodal RFM analysis and deep learning. Background Technology

[0002] In today's business environment, customer segmentation and precision marketing are core to enhancing a company's competitiveness. However, existing customer segmentation methods still have many limitations: The data dimensions are limited, resulting in a one-sided customer profile. Existing technologies largely rely on traditional structured data, such as the Recency, Frequency, Monetary Model (RFM) based on transaction records. While the RFM model is simple and effective, it completely ignores unstructured customer data, such as the emotional tone of voice during customer service calls, and visual features from ID photos or behavioral images.

[0003] Clustering methods suffer from poor interpretability and difficulty in business implementation. While traditional machine learning clustering methods (such as K-Means) are computationally efficient, their clustering results are like a "black box," lacking clear explanations of business rules. Business personnel cannot understand "why these customers are grouped together," making it difficult to effectively integrate the clustering results with specific marketing actions (such as sending specific coupons or conducting follow-up phone calls), thus reducing the model's practical value. Summary of the Invention

[0004] The purpose of this application is to address the problems of existing analytical methods, such as the limited dimensions of customer profiling leading to one-sided customer insights and the poor interpretability of customer segmentation methods, which makes business implementation difficult.

[0005] Customer segmentation methods that integrate multimodal RFM analysis and deep learning include: Multimodal data acquisition and preprocessing: Speech, image, and structured data are acquired through multiple channels, and then the acquired data is preprocessed based on quantile-based RFM data normalization. The original RFM (Recency, Frequency, Monetary) indicators are standardized using a quartile-based 3-point system, converting the original indicators into standardized scores of 0-3, and then filtering out customer data that conform to the standardized feature vector. Multimodal feature extraction and fusion: Deep learning models are used to extract features from preprocessed data of different modalities. These feature vectors are then concatenated to form a unified high-dimensional multimodal fusion feature vector to build a comprehensive and three-dimensional customer profile. A multi-indicator fusion framework for determining the K value: Before K-means clustering, a multi-indicator fusion, stability-driven, and business-oriented framework for determining the number of clusters K is introduced. This framework comprehensively evaluates internal indicators, theoretical indicators, and stability indicators, and normalizes and weights these indicators to select the K value with the highest comprehensive score as the optimal number of clusters.

[0006] Furthermore, multi-channel data collection includes: collecting customer service call recordings from call centers as voice data; obtaining customer ID photos and behavioral-related images from real-name authentication systems, authorized apps, and offline stores as image data; and extracting customer RFM data, basic information, and product purchase records from the enterprise's business database as structured data.

[0007] Furthermore, the original RFM calculates three metrics for each customer based on the raw data: = Analysis cutoff date - Client's last transaction date; = Total number of transactions by the customer within the selected period; = Total spending by customers within the selected period; in, , and These represent the most recent purchase, purchase frequency, and purchase amount, respectively.

[0008] Furthermore, the 3-point standardization process for quartiles is as follows: Standardized score = 3 × (original index - Q1) / (Q3 - Q1); Where Q1 is the first quartile (25th percentile) and Q3 is the third quartile (75th percentile); Specifically, for each indicator: A) Recency(R) standardization: in, This represents the raw Recency data, as Recency exhibits a negative characteristic (i.e., the fewer the transaction interval days, the higher the customer value). Indicates to The values ​​are positively oriented to achieve a consistent characteristic where larger values ​​are better. and These represent the results after forwarding. The first and third quartiles of the data.

[0009] B) Frequency (F) standardization: in, This represents the frequency-normalized feature vector that meets the requirements of the K-means clustering algorithm. and These represent the first and third quartiles in the original frequency data, respectively.

[0010] C) Monetary(M) standardization: in, This represents the Monetary-normalized feature vector that meets the requirements of the K-means clustering algorithm. and These represent the first and third quartiles in the original Monetary data, respectively.

[0011] Then, values ​​exceeding the interval [0,3] are truncated to the boundary. RFM data is modeled into standardized feature vectors that meet the requirements of the K-means clustering algorithm. Based on three customer indicator parameters, customers are defined as: champion customers, loyal customers, potential customers, customers to be retained, customers at risk of churn, new customers, and dormant customers. Furthermore, the feature extraction specifically involves: extracting speech emotion features using a convolutional recurrent neural network (CRNN), extracting image visual features using a pre-trained CNN model, and extracting structured behavioral features using a fully connected neural network. Furthermore, the acquisition of internal indicators, theoretical indicators, and stability indicators is specifically as follows: Set the range of candidate K values ​​and basic clustering: Based on prior business knowledge or data scale, set a reasonable range of K values ​​for search. For each candidate K value within this range, execute the K-means clustering algorithm and record the clustering results. For each candidate K value, three types of evaluation indicators are calculated to form a comprehensive evaluation; I) Internal metrics: These assess cluster density and separation by calculating the silhouette coefficient for each candidate K value to evaluate intra-cluster density and inter-cluster separation. This process can be modeled as follows: Wherein, SC represents the silhouette coefficient of the entire dataset, which is the average silhouette coefficient of all samples. The value of SC ranges from [-1, 1]. A larger value indicates a better clustering effect. This represents the silhouette coefficient for each sample, where n represents the total number of customers in the entire dataset. in, and Representing samples respectively The average distance to all other samples in the same cluster and the sample The average distance to all samples in the nearest neighbor cluster; in, It is a sample The cluster to which it belongs It is the size of the cluster. It is a sample and The distance between them It is different Other clusters; II) Theoretical Indicators: To assess the statistical significance of the K value compared to random data, the Gap statistic is calculated to evaluate the statistical significance of the K value compared to random data. First, the intra-cluster dispersion of the actual data is calculated: in, This represents the intra-cluster dispersion of the current candidate k value. It is a cluster The sum of distances between all sample pairs within the range. It is a cluster Size; Then, B reference datasets are generated (usually generated using a uniform distribution), and for each reference dataset... Calculate its intra-cluster dispersion And calculate the gap statistic for the current candidate k value: in, This represents the intra-cluster scatter of the current candidate k value and the b-th uniformly distributed dataset.

[0012] III) Stability Metric: The consensus clustering score assesses the robustness of the clustering results. It generates D resampled datasets by sampling the original dataset with replacement. Each resampled dataset is then clustered, and a consensus matrix is ​​constructed to record the frequency with which each pair of samples is assigned to the same cluster in the D clustering operations. elements Indicates sample and the proportion of individuals assigned to the same cluster in multiple clustering processes: Finally, the average consensus value of the consensus matrix is ​​calculated: in, represents the average consensus value of the consensus matrix, n represents the total number of all customers (sample points) in the entire dataset, and 2 / n(n-1) represents the number of all unique sample pairs in the dataset.

[0013] Furthermore, the normalization and weighted fusion are specifically as follows: For each indicator, its original score within the candidate K value range is subjected to minimum-maximum normalization, mapping it to the [0,1] interval. Subsequently, a weighted comprehensive scoring model is constructed, assigning appropriate weights to the indicators of each dimension, and calculating the comprehensive score for each K value. in, It is an internal indicator dimension score. It is a score based on theoretical indicator dimensions. It is the score in the stability index dimension. These represent the relative importance of the three evaluation dimensions, and .

[0014] Finally, the K value with the highest comprehensive score is selected as the optimal number of clusters. This formula, through systematic multi-dimensional weighted evaluation, ensures that the final selected number of clusters is mathematically optimal, statistically significant, and robust and reliable in actual business.

[0015] The specific results of performing K-means clustering are as follows: using the determined optimal K value, the K-means clustering algorithm is run on the customer's RFM feature vector. After the clustering is completed, the centroid (mean) of each cluster in the three dimensions of R, F, and M is analyzed. The clustering results output includes: the cluster label to which each customer group belongs, as well as the RFM feature center point and clear business definition for each cluster.

[0016] A customer segmentation system integrating multimodal RFM analysis and deep learning, including: Multimodal data acquisition and preprocessing module: responsible for data input and standardization processing, providing a high-quality data source for subsequent feature extraction; Multimodal feature extraction and fusion module: This module uses a deep learning model to extract features from preprocessed data of different modalities, and then concatenates these feature vectors to form a unified high-dimensional multimodal fusion feature vector to build a comprehensive and three-dimensional customer profile. Multi-indicator fusion K-value determination module: responsible for providing the optimal K-value based on business needs, solving the problem of subjective K-value selection in traditional K-means clustering.

[0017] Furthermore, the multimodal data acquisition and preprocessing module includes: Multi-channel data acquisition unit: integrates business system API interfaces, voice acquisition module (microphone array + noise reduction circuit), image acquisition unit (camera + real-time transmission module), and third-party data access interface to realize voice, image, and structured data; RFM indicator standardization unit: stores quartile division rules (pre-calculate Q1, Q2, Q3 quantiles based on historical customer data), and has a built-in 3-point mapping algorithm to compare the original RFM indicators (spending amount M, spending frequency F, recent spending time R) with the quantiles respectively, and output the standardized score in the interval [0,3]. Data filtering unit: It presets the feature vector validity threshold, filters the generated RFM feature vectors and associated multimodal data, retains the customer data packets that meet the conditions, and transmits them to the feature extraction module.

[0018] Furthermore, the multimodal feature extraction and fusion module includes: Single-modal feature extraction unit: It contains three dedicated feature extraction sub-modules: speech feature sub-module, image feature sub-module, and structured feature sub-module. Each sub-module has a built-in model training interface and supports parameter updates.

[0019] Multimodal feature fusion unit: Using a feature concatenation strategy, the three types of feature vectors with unified dimensions are concatenated in sequence to generate a high-dimensional multimodal fusion feature vector.

[0020] Furthermore, the multi-indicator fusion K-value determination module includes: Multi-dimensional indicator calculation unit: Includes three types of indicator calculation sub-modules: internal indicator sub-module (calculates silhouette coefficient to evaluate cluster cohesion and separation), theoretical indicator sub-module (assesses the statistical significance of K value compared with random data by calculating Gap statistic), and stability indicator sub-module (generates D resampled datasets by sampling the original dataset with replacement (e.g., Bootstrap), clusters each resampled dataset, constructs a consensus matrix, records the frequency of each pair of samples being assigned to the same cluster in D clusters, and calculates the average consensus value of the consensus matrix; the higher the value, the more stable the clustering result). Indicator normalization and weighting unit: The original score within the candidate K value range is normalized by minimum-maximum and mapped to the [0,1] interval; Built-in weight configuration interface, which supports setting the weight coefficients of three types of indicators according to business scenarios (such as improving the weight of stability indicators in precision marketing scenarios), and calculates the comprehensive score of each candidate K value (usually K∈[2,10]) by weighted summation; The optimal K-value selection unit iterates through the comprehensive scores of candidate K-values ​​and selects the K-value with the highest score as the optimal number of clusters; it also has a built-in score visualization sub-unit that outputs individual indicators and comprehensive score curves for each K-value, providing a basis for business decisions. Beneficial effects

[0021] This invention first employs multimodal feature extraction and fusion technology to comprehensively extract features from customer speech, images, and structured data. Specifically, speech data is processed using a convolutional recurrent neural network (CRNN) to extract the customer's emotional state; image data is processed using a pre-trained CNN model to extract visual attributes such as age and gender; and structured data (such as RFM) is processed using a fully connected neural network to extract high-order behavioral features. These feature vectors from different modalities are concatenated into a unified high-dimensional fused feature vector, thereby constructing a comprehensive and three-dimensional customer profile encompassing behavioral, emotional, and identity features. This greatly enriches the dimensions and depth of customer insights and solves the problem of one-sided profiles caused by traditional single data sources. This invention employs a K-value determination framework based on quantile-based RFM data standardization and multi-indicator fusion. Before K-means clustering, the original RFM data is first standardized using a quartile-based 3-point scale, transforming the complex original indicators into simple scores of 0-3. This allows the RFM feature intervals of each customer group to directly correspond to clear business definitions (e.g., "champion customer" corresponds to high R, high F, and high M), thereby greatly improving the interpretability of the clustering results. Simultaneously, to avoid the limitations of traditional K-value determination methods, this solution introduces a multi-indicator fusion framework for K-value determination. This framework comprehensively considers the internal compactness and separation of the clustering results (evaluated by silhouette coefficient), statistical significance (evaluated by Gap statistic), and robustness of the results (evaluated by consensus clustering score), and weights and fuses these indicators to ultimately determine a mathematically optimal, statistically significant, and business-robust number of clusters K. Finally, the optimal K value is used to perform K-means clustering on the fused multimodal feature vectors, and the standardized RFM centroids are combined to provide a clear business description for each customer group. This makes the clustering results not only accurate but also highly interpretable, making it easier for business personnel to understand and implement specific marketing strategies. Attached Figure Description

[0022] The invention will now be further described with reference to the accompanying drawings; Figure 1 This is a schematic diagram of the method flow proposed in this invention; Figure 2 This is a schematic diagram of the system structure of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0025] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying a significant factor.

[0026] Example 1 refer to Figure 1 - Figure 2 This paper discloses a customer segmentation method integrating multimodal RFM analysis and deep learning, including: multimodal data collection and preprocessing: collecting voice, image, and structured data through multiple channels, including: collecting customer service call recordings from call centers as voice data; obtaining customer ID photos and behavior-related images from real-name authentication systems, authorized apps, and offline stores as image data; and extracting customer RFM data, basic information, and product purchase records from enterprise business databases as structured data. To make the RFM data more suitable for the distance sensitivity requirements of the K-means clustering algorithm and improve the clustering effect, the collected data is preprocessed based on quantile-based RFM data standardization, based on the original index calculation. The original RFM (Recency, Frequency, Monetary) metrics are standardized using a quartile-based 3-point system, converting them into standardized scores of 0-3. This process filters out customer data that conforms to the standardized feature vectors, ensuring that the RFM data is more suitable for K-means clustering and that the clustering results have clear business interpretability. The original RFM calculates three metrics for each customer based on the raw data: = Analysis cutoff date - Client's last transaction date; = Total number of transactions by the customer within the selected period; = Total spending by customers within the selected period; in, , and These represent the most recent purchase, purchase frequency, and purchase amount, respectively.

[0027] Subsequently, to facilitate interpretability analysis of customer segmentation using the K-means clustering algorithm, we standardized the above indicators based on quantiles. Specifically, we used quartiles for standardization, and the quartile standardization process (3-point scale) is as follows: Standardized score = 3 × (original index - Q1) / (Q3 - Q1); Where Q1 is the first quartile (25th percentile) and Q3 is the third quartile (75th percentile); Specifically, for each indicator: A) Recency(R) standardization: in, This represents the raw Recency data, as Recency exhibits a negative characteristic (i.e., the fewer the transaction interval days, the higher the customer value). Indicates to The values ​​are positively oriented to achieve a consistent characteristic where larger values ​​are better. and These represent the results after forwarding. The first and third quartiles of the data.

[0028] B) Frequency (F) standardization: in, This represents the frequency-normalized feature vector that meets the requirements of the K-means clustering algorithm. and These represent the first and third quartiles in the original frequency data, respectively.

[0029] C) Monetary(M) standardization: in, This represents the Monetary-normalized feature vector that meets the requirements of the K-means clustering algorithm. and These represent the first and third quartiles in the original Monetary data, respectively.

[0030] Then, values ​​exceeding the [0,3] interval are truncated to the boundary to avoid the impact of outliers on the overall distribution.

[0031] This method models RFM data into standardized feature vectors that meet the requirements of the K-means clustering algorithm, and the 3-point standardization ensures clear interpretability and comparability among different customer groups. Specifically, customer segmentation based on 3-point RFM data has the following clear business definitions, direct strategic guidance, and concise group profiles:

[0032] Multimodal Feature Extraction and Fusion: Speech features are extracted using a Convolutional Recurrent Neural Network (CRNN). This process first converts the speech into a spectrogram, then uses a CNN to extract local features, followed by a recurrent neural network (RNN) to capture temporal dependencies, ultimately outputting an emotional feature vector reflecting the customer's emotional state. Image features are extracted using a pre-trained CNN model (such as ResNet). By removing the final classification layer and extracting features before the fully connected layer, feature vectors representing visual attributes such as age and gender are obtained. Simultaneously, structured data is input into a fully connected neural network to learn and output a high-order behavioral feature vector. Finally, the emotional, image, and behavioral feature vectors are concatenated to form a unified high-dimensional multimodal fusion feature vector, which serves as the input for subsequent customer segmentation.

[0033] A Multi-Indicator Fusion Framework for Determining the K-Value: Customer Segmentation Based on a Multi-Indicator Fusion K-Means Clustering Algorithm. In the field of customer segmentation, determining the optimal number of customer groups (cluster number K) is a core prerequisite for ensuring the quality of segmentation, but existing technologies suffer from serious systemic defects in this crucial step. Existing technologies typically rely on a single indicator in isolation (such as observing the inflection point solely through the "elbow rule" or pursuing only the maximization of the "profile coefficient") to determine the K-value. This approach ignores the inherent principles and limitations of different evaluation indicators. For example, the "inflection point" judgment of the elbow rule is highly dependent on subjective human identification, and different analysts may reach different conclusions; while the profile coefficient is not effective in evaluating non-convex, streamlined cluster structures. Before K-means clustering, a multi-indicator fusion, stability-driven, and business-oriented framework for determining the number of clusters K is introduced, fundamentally solving the above problems and bringing multi-dimensional improvements. This framework comprehensively evaluates internal indicators, theoretical indicators, and stability indicators, and normalizes and weights these indicators to construct a comprehensive evaluation system. This system can comprehensively verify candidate K values ​​from multiple dimensions such as intra-cluster compactness, inter-cluster separation, statistical significance, and robustness of results, thereby arriving at a consensus and finally selecting the K value with the highest comprehensive score as the optimal number of clusters, so as to ensure the quality, statistical significance, and robustness of the clustering results.

[0034] The internal indicators, theoretical indicators, and stability indicators are obtained as follows: Set the candidate K value range and basic clustering: Based on prior business knowledge or data scale, set a reasonable K value search range, such as K=[2,10]. For each candidate K value within this range, execute the K-means clustering algorithm and record the clustering results (cluster label and cluster center of each sample). For each candidate K value, three types of evaluation indicators are calculated to form a comprehensive evaluation; I) Internal metrics: These assess cluster density and separation by calculating the silhouette coefficient for each candidate K value to evaluate intra-cluster density and inter-cluster separation. This process can be modeled as follows: Wherein, SC represents the silhouette coefficient of the entire dataset, which is the average silhouette coefficient of all samples. The value of SC ranges from [-1, 1]. A larger value indicates a better clustering effect. This represents the silhouette coefficient for each sample, where n represents the total number of customers in the entire dataset. in, and Representing samples respectively The average distance to all other samples in the same cluster and the sample The average distance to all samples in the nearest neighbor cluster; in, It is a sample The cluster to which it belongs It is the size of the cluster. It is a sample and The distance between them It is different Other clusters; II) Theoretical Indicators: To assess the statistical significance of the K value compared to random data, the gap statistic for different candidate k values ​​is calculated for subsequent comprehensive score calculation. That is, the statistical significance of the K value compared to random data is evaluated by calculating the gap statistic. The larger the gap statistic, the more significantly the clustering structure outperforms the random distribution. First, the intra-cluster dispersion of the actual data is calculated: in, This represents the intra-cluster dispersion of the current candidate k value, and r represents the traversal variable for the current candidate k value, iterating through all clusters from 1 to k. Here, k refers to the current candidate k value. A reasonable search range for k values ​​is set based on prior knowledge or data size. For each candidate k value within this range, the K-means clustering algorithm is executed, and the clustering results are recorded. Finally, the gap statistic is calculated here by traversing all candidate k values ​​for subsequent calculation of the overall score. It is a cluster The sum of distances between all sample pairs within the range. It is a cluster Size; Subsequently, B reference datasets are generated, where B represents the number of datasets with the same sample size generated through a uniform distribution. For each reference dataset... Calculate its intra-cluster dispersion Let b represent the b-th such dataset, i.e., b takes the value [1, B]. Calculate the gap statistic for the current candidate k value: in, This represents the intra-cluster scatter of the current candidate k value and the b-th uniformly distributed dataset.

[0035] III) Stability Metric: The consensus clustering score assesses the robustness of the clustering results. It is achieved by sampling the original dataset with replacement (e.g., Bootstrap) to generate D (e.g., D=100) resampled datasets. Each resampled dataset is then clustered, and a consensus matrix is ​​constructed to record the frequency with which each pair of samples is assigned to the same cluster in the D clustering iterations. elements Indicates sample and The proportion of individuals assigned to the same cluster across multiple clustering sessions: Finally, the average consensus value of the consensus matrix is ​​calculated. The higher the value, the more stable the clustering results. in, represents the average consensus value of the consensus matrix, n represents the total number of all customers (sample points) in the entire dataset, and 2 / n(n-1) represents the number of all unique sample pairs in the dataset.

[0036] Normalization and weighted fusion are specifically performed as follows: Since different indicators have different dimensions and meanings, normalization is required, followed by weighted fusion. For each indicator, its original score within the candidate K value range is minimum-maximum normalized, mapping it to the [0,1] interval. Subsequently, a weighted comprehensive scoring model is constructed, assigning appropriate weights to indicators in each dimension, and calculating the comprehensive score for each K value. in, It is an internal indicator dimension score. It is a score based on theoretical indicator dimensions. It is the score in the stability index dimension. These represent the relative importance of the three evaluation dimensions, and .

[0037] Finally, the K value with the highest comprehensive score is selected as the optimal number of clusters. This formula, through systematic multi-dimensional weighted evaluation, ensures that the final selected number of clusters is mathematically optimal, statistically significant, and robust and reliable in actual business.

[0038] The specific results of K-means clustering are as follows: using a determined optimal K value, the K-means clustering algorithm is run on the customer's RFM feature vectors. After clustering, the centroids (means) of each cluster are analyzed in the three dimensions of R, F, and M, thereby assigning a clear business meaning to each cluster, such as: I) Cluster A (New Customers): High R value (recent purchase), but low F and M values. Strategy Focus: Nurturing and guiding, converting them into loyal customers through personalized content and support.

[0039] II) Cluster B (Champion Customers): High R-value, high F-value, and high M-value. Strategy Focus: Maintenance and rewards, providing exclusive offers, priority service, and new product previews to maximize their lifetime value.

[0040] Cluster C (dormant customers): Low R value (long inactive), moderate or fluctuating F and M values. Strategy focus: Reactivation and re-engagement, rebuilding connections through targeted promotions and reactivation campaigns.

[0041] The clustering results output includes: the cluster label to which each customer group belongs, as well as the RFM feature center point and clear business definition of each cluster, providing a direct and reliable basis for subsequent sales forecasting and precision marketing.

[0042] Example 2 Reference Figure 2 A publicly available customer segmentation system integrating multimodal RFM analysis and deep learning, including: Multimodal data acquisition and preprocessing module: responsible for data input and standardization processing, providing a high-quality data source for subsequent feature extraction; Multimodal feature extraction and fusion module: This module uses a deep learning model to extract features from preprocessed data of different modalities, and then concatenates these feature vectors to form a unified high-dimensional multimodal fusion feature vector to build a comprehensive and three-dimensional customer profile. Multi-indicator fusion K-value determination module: responsible for providing the optimal K-value based on business needs, solving the problem of subjective K-value selection in traditional K-means clustering.

[0043] The multimodal data acquisition and preprocessing module includes: Multi-channel data acquisition unit: integrates business system API interfaces, voice acquisition module (microphone array + noise reduction circuit), image acquisition unit (camera + real-time transmission module), and third-party data access interface to realize voice, image, and structured data; RFM indicator standardization unit: stores quartile division rules (pre-calculate Q1, Q2, Q3 quantiles based on historical customer data), and has a built-in 3-point mapping algorithm to compare the original RFM indicators (spending amount M, spending frequency F, recent spending time R) with the quantiles respectively, and output the standardized score in the interval [0,3]. Data filtering unit: It presets the feature vector validity threshold, filters the generated RFM feature vectors and associated multimodal data, retains the customer data packets that meet the conditions, and transmits them to the feature extraction module.

[0044] The multimodal feature extraction and fusion module includes: Single-modal feature extraction unit: It contains three dedicated feature extraction sub-modules: speech feature sub-module, image feature sub-module, and structured feature sub-module. Each sub-module has a built-in model training interface and supports parameter updates.

[0045] Multimodal feature fusion unit: Using a feature concatenation strategy, the three types of feature vectors with unified dimensions are concatenated in sequence to generate a high-dimensional multimodal fusion feature vector.

[0046] The multi-indicator fusion K-value determination module includes: Multi-dimensional indicator calculation unit: Includes three types of indicator calculation sub-modules: internal indicator sub-module (calculates silhouette coefficient to evaluate cluster cohesion and separation), theoretical indicator sub-module (assesses the statistical significance of K value compared with random data by calculating Gap statistic), and stability indicator sub-module [by performing sampling with replacement (such as Bootstrap) on the original dataset to generate D resampled datasets, clustering each resampled dataset, and constructing a consensus matrix to record the frequency of each pair of samples being assigned to the same cluster in D clusters, and calculating the average consensus value of the consensus matrix. The higher the value, the more stable the clustering result.] Indicator normalization and weighting unit: The original score within the candidate K value range is normalized by minimum-maximum and mapped to the [0,1] interval; Built-in weight configuration interface, which supports setting the weight coefficients of three types of indicators according to business scenarios (such as improving the weight of stability indicators in precision marketing scenarios), and calculates the comprehensive score of each candidate K value (usually K∈[2,10]) by weighted summation; The optimal K-value selection unit iterates through the comprehensive scores of candidate K-values ​​and selects the K-value with the highest score as the optimal number of clusters; it also has a built-in score visualization sub-unit that outputs individual indicators and comprehensive score curves for each K-value, providing a basis for business decisions.

[0047] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0048] The above formulas are all derived from software simulation using a large amount of data, and are selected to be close to the actual values. The coefficients in the formulas are set by those skilled in the art based on the actual situation. The above are only preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or changes made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A customer segmentation method fusing multi-modal RFM analysis with deep learning, characterized in that, include: Multimodal data acquisition and preprocessing: Speech, image and structured data are acquired through multiple channels, and then the acquired data is preprocessed using RFM data standardization based on quantiles; Multimodal feature extraction and fusion: Deep learning models are used to extract features from preprocessed data of different modalities. These feature vectors are then concatenated to form a unified high-dimensional multimodal fusion feature vector to build a comprehensive and three-dimensional customer profile. A multi-indicator fusion framework for determining the K value: Before K-means clustering, a multi-indicator fusion, stability-driven, and business-oriented framework for determining the number of clusters K is introduced. This framework comprehensively evaluates internal indicators, theoretical indicators, and stability indicators, and normalizes and weights these indicators to select the K value with the highest comprehensive score as the optimal number of clusters.

2. The method of claim 1, wherein the method of fusing multi-modal RFM analysis with deep learning for customer segmentation is characterized by, Multi-channel data collection includes: collecting customer service call recordings from call centers as voice data; obtaining customer ID photos and behavior-related images from real-name authentication systems or authorized apps and offline stores as image data; and extracting customer RFM data, basic information and product purchase records from the enterprise's business database as structured data. The standardization preprocessing specifically involves: standardizing the original RFM indicators using a quartile-based 3-point system, converting the original indicators into standardized scores of 0-3, and filtering out customer data that conform to the standardized feature vector.

3. The method of claim 2, wherein the method of fusing multi-modal RFM analysis with deep learning for customer segmentation is characterized by, The original RFM calculates three metrics for each customer based on the raw data: = Analysis cutoff date - Client's last transaction date; = Total number of transactions by the customer within the selected period; = Total spending by customers within the selected period; in, , and These represent the most recent purchase, purchase frequency, and purchase amount, respectively.

4. The customer segmentation method integrating multimodal RFM analysis and deep learning according to claim 3, characterized in that, The 3-point standardization process for quartiles is as follows: Standardized score = 3 × (original index - Q1) / (Q3 - Q1); Where Q1 is the first quartile and Q3 is the third quartile; Specifically, for each indicator: A) Recency(R) standardization: in, This represents the raw Recency data, which exhibits a negative characteristic. Indicates to The values ​​are positively oriented to achieve a consistent characteristic where larger values ​​are better. and These represent the results after forwarding. The first and third quartiles of the data; B) Frequency (F) standardization: in, This represents the frequency-normalized feature vector that meets the requirements of the K-means clustering algorithm. and These represent the first and third quartiles in the original Frequency data, respectively. C) Monetary(M) standardization: in, This represents the Monetary-normalized feature vector that meets the requirements of the K-means clustering algorithm. and These represent the first and third quartiles in the original Monetary data, respectively. Then, values ​​exceeding the [0,3] interval are truncated to the boundary, and the RFM data is modeled into a standardized feature vector that meets the requirements of the K-means clustering algorithm.

5. The customer segmentation method integrating multimodal RFM analysis and deep learning according to claim 1, characterized in that, The feature extraction process specifically involves: extracting speech emotion features through a convolutional recurrent neural network (CRNN), extracting image visual features through a pre-trained CNN model, and extracting structured behavioral features through a fully connected neural network.

6. The customer segmentation method integrating multimodal RFM analysis and deep learning according to claim 1, characterized in that, The internal indicators, theoretical indicators, and stability indicators are obtained as follows: Set the range of candidate K values ​​and basic clustering: Based on prior business knowledge or data scale, set a reasonable range of K values ​​for search. For each candidate K value within this range, execute the K-means clustering algorithm and record the clustering results. For each candidate K value, three types of evaluation indicators are calculated to form a comprehensive evaluation; I) Internal metrics: These assess cluster density and separation by calculating the silhouette coefficient (SC) for each candidate K value. This process can be modeled as follows: Where SC represents the silhouette coefficient of the entire dataset, and the value of SC ranges from [-1, 1]. This represents the silhouette coefficient for each sample, where n represents the total number of customers in the entire dataset. in, and Representing samples respectively The average distance to all other samples in the same cluster and the sample The average distance to all samples within the nearest cluster; in, It is a sample The cluster to which it belongs It is the size of the cluster. It is a sample and The distance between them It is different Other clusters; II) Theoretical Indicators: To assess the statistical significance of the K value compared to random data, the Gap statistic is calculated to evaluate the statistical significance of the K value compared to random data. First, the intra-cluster dispersion of the actual data is calculated: in, This represents the intra-cluster dispersion of the current candidate k value. It is a cluster The sum of distances between all sample pairs within the range. It is a cluster Size; Then B reference datasets are generated, and for each reference dataset... Calculate its intra-cluster dispersion And calculate the gap statistic for the current candidate k value: in, This represents the intra-cluster scatter of the current candidate k value and the b-th uniformly distributed dataset; III) Stability Metric: The consensus clustering score assesses the robustness of the clustering results. It generates D resampled datasets by sampling the original dataset with replacement. Each resampled dataset is then clustered, and a consensus matrix is ​​constructed to record the frequency with which each pair of samples is assigned to the same cluster in the D clustering operations. elements Indicates sample and The proportion of individuals assigned to the same cluster across multiple clustering sessions: Finally, the average consensus value of the consensus matrix is ​​calculated: in, represents the average consensus value of the consensus matrix, n represents the total number of all customers in the entire dataset, and 2 / n(n-1) represents the number of all unique sample pairs in the dataset.

7. The customer segmentation method integrating multimodal RFM analysis and deep learning according to claim 6, characterized in that, Normalization and weighted fusion are specifically performed as follows: For each indicator, its original score within the candidate K value range is minimum-maximum normalized to map it to the [0,1] interval. Subsequently, a weighted comprehensive scoring model is constructed, assigning appropriate weights to indicators in each dimension, and calculating the comprehensive score for each K value. in, It is an internal indicator dimension score. It is a score based on theoretical indicator dimensions. It is the score in the stability index dimension. These represent the relative importance of the three evaluation dimensions, and ; Finally, the K value with the highest overall score is selected as the optimal number of clusters. This formula is evaluated through a systematic multi-dimensional weighted assessment. The specific results of performing K-means clustering are as follows: using the determined optimal K value, the K-means clustering algorithm is run on the customer's RFM feature vector. After the clustering is completed, the centroid of each cluster in the three dimensions of R, F, and M is analyzed. The clustering results output includes: the cluster label to which each customer group belongs, as well as the RFM feature center point and clear business definition for each cluster.

8. A customer segmentation system integrating multimodal RFM analysis and deep learning, the system being applicable to the method described in any one of claims 1-7, characterized in that, include: Multimodal data acquisition and preprocessing module: responsible for data input and standardization processing, providing a high-quality data source for subsequent feature extraction; Multimodal feature extraction and fusion module: This module uses a deep learning model to extract features from preprocessed data of different modalities, and then concatenates these feature vectors to form a unified high-dimensional multimodal fusion feature vector to build a comprehensive and three-dimensional customer profile. Multi-indicator fusion K-value determination module: responsible for providing the optimal K-value based on business needs, solving the problem of subjective K-value selection in traditional K-means clustering.

9. The customer segmentation system integrating multimodal RFM analysis and deep learning according to claim 8, characterized in that, The multimodal data acquisition and preprocessing module includes: Multi-channel data acquisition unit: integrates business system API interfaces, voice acquisition modules, image acquisition units, and third-party data access interfaces to achieve the acquisition of voice, image, and structured data; RFM index standardization unit: stores quartile division rules, has a built-in 3-point mapping algorithm, compares the original RFM index with the quantiles respectively, and outputs the standardized score in the interval [0,3]. Data filtering unit: It presets the feature vector validity threshold, filters the generated RFM feature vectors and associated multimodal data, retains the customer data packets that meet the conditions, and transmits them to the feature extraction module.

10. The customer segmentation system integrating multimodal RFM analysis and deep learning according to claim 9, characterized in that, The multi-indicator fusion K-value determination module includes: Multi-dimensional indicator calculation unit: includes three types of indicator calculation sub-modules: internal indicator sub-module, theoretical indicator sub-module, and stability indicator sub-module; Indicator normalization and weighting unit: The original score within the candidate K value range is normalized by minimum-maximum and mapped to the [0,1] interval; Built-in weight configuration interface, which supports setting the weight coefficients of the three types of indicators according to the business scenario, and calculates the comprehensive score of each candidate K value by weighted summation; The optimal K-value selection unit iterates through the comprehensive scores of candidate K-values ​​and selects the K-value with the highest score as the optimal number of clusters; it also has a built-in score visualization sub-unit that outputs individual indicators and comprehensive score curves for each K-value.

Images