Bitmap-driven efficient customer subdivision circling device and method

Abstract

The invention relates to the field of data processing, and provides a bitmap-driven high-efficiency customer subdivision circling device and a bitmap-driven high-efficiency customer subdivision circling method. The method aims at solving the problems that in a traditional client labeling method, performance, maintainability and readability are poor, flexibility is limited, data consistency is difficult to guarantee, version control is lacked, and complex logic is difficult to achieve. According to the method, client data is collected from a plurality of data sources, cleaned, converted and integrated, and then the data is stored in a bitmap to construct a data model. By extracting key features and indexes, a comprehensive portrait is constructed for each customer, and the portraits are utilized to define a circle group division rule. And finally, determining a circle group to which each customer belongs through bitmap operation, and storing a result in a bitmap. The method has remarkable advantages in the aspects of data processing efficiency, storage optimization, customer segmentation precision, business application flexibility and the like, and can be widely applied to scenes of precision marketing, customer segmentation, product recommendation and the like.

Description

Technical Field

[0001] The invention relates to the field of data processing, and provides a high-efficiency customer segmentation selection device and method driven by a bitmap. Background Art

[0002] In today's business environment, customer segmentation and precision marketing are becoming increasingly important. In order to better understand and serve customers, companies need to classify and analyze customers so as to adopt targeted marketing strategies. Traditional customer segmentation methods usually rely on SQL queries and multi-table joins, which have performance bottlenecks and insufficient flexibility in large-scale data processing.

[0003] Current technical solutions include: Traditional customer labeling method: Enterprises usually determine key features or indicators based on business needs, such as consumption amount, purchase frequency, age, gender, etc., extract relevant data from the database or business system, and filter and classify the data according to the set thresholds or rules. For example, customers whose consumption amount exceeds a certain value are marked as "high-value customers", and customers whose purchase frequency is less than a certain number of times are marked as "low-frequency buyers". This method has performance problems, maintainability and readability problems, limited flexibility, difficulty in ensuring data consistency, lack of version control, and difficulty in implementing complex logic.

[0004] Customer segmentation based on machine learning: Some companies have begun to use machine learning algorithms to analyze customer data to achieve more refined customer segmentation. However, this method usually requires exporting data to external systems for processing, which increases the complexity of data processing and the cumbersomeness of the process.

[0005] Tag-based customer segmentation method: Some companies try to use tags to represent different characteristics and behaviors of customers, and define different customer segments through the combination of tags. However, this method usually requires complex SQL queries and multi-table joins, and has performance bottlenecks and insufficient flexibility in large-scale data processing.

[0006] Therefore, the existing technology has performance bottlenecks, insufficient flexibility, maintainability and readability problems, difficulty in ensuring data consistency, lack of version control, and difficulty in implementing complex logic when processing large-scale data. The present invention aims to solve these problems and provide a special customer group selection method and storage medium, which can realize the rapid processing and accurate selection of large-scale data through the efficient storage and computing capabilities of Bitmap. Summary of the invention

[0007] The purpose of the present invention is to solve the problems of performance, maintainability and readability, limited flexibility, difficulty in ensuring data consistency, lack of version control and difficulty in implementing complex logic in traditional client labeling methods.

[0008] In order to achieve the above purpose, the present invention adopts the following technical solutions: The present invention provides an efficient customer segmentation and selection device driven by a bitmap, comprising: Data collection module, used to collect customer-related data from multiple data sources and clean, transform and integrate them; A bitmap storage module is used to store the integrated data in a bitmap and construct a data model; Feature extraction module, which is used to extract key features and indicators based on stored data to build a comprehensive profile for each customer; The circle group division module is used to use customer portraits to clarify the rules for dividing circles; A circle group determination module is used to determine the circle group to which each customer belongs by querying and calculating in the bitmap using defined circle group rules; The result storage module is used to store the circle group results in a bitmap for subsequent query and use.

[0009] Further, the data collection module includes: The data type identification submodule is used to identify and determine the types of customer-related data that need to be collected, including basic user information, consumption behavior data, and interactive behavior data; The data source determination submodule is used to determine the data source, including internal data sources and external data sources. The internal data sources include the order system, CRM system, and website access logs, and the external data sources include social media data and third-party data services. The data collection submodule is used to select appropriate data collection tools and technologies for data collection according to the characteristics of the data source; The data cleaning submodule is used to clean the collected data, including removing duplicate data, filling missing values, and correcting erroneous data; The data conversion submodule is used to convert the cleaned data, unify the data formats of different data sources into a standard format, and convert the data type into a format suitable for bitmap storage; A data integration submodule is used to integrate the converted data into a unified data storage system; The quality check submodule is used to perform quality checks on the integrated data to ensure the accuracy and consistency of the data; The data classification and labeling submodule is used to classify and label data according to business needs.

[0010] Further, the bitmap storage module includes: A bitmap structure determination submodule is used to determine the bitmap storage structure, including selecting a suitable bitmap implementation method to optimize storage space and query performance; A data mapping submodule, which is used to map customer-related data into a bitmap structure, assign a unique identifier to each customer, and map the customer's characteristics and indicators to different bits of the bitmap; The data model building submodule is used to build data models according to business requirements, including creating customer dimension tables and behavior fact tables; A bitmap data compression submodule is used to compress bitmap data to save storage space and improve query efficiency; The data indexing submodule is used to index the bitmap data according to the definition of the data model so as to quickly locate and query the data of a specific customer; The data storage submodule is used to store the constructed bitmap data model in a suitable data storage system.

[0011] Furthermore, the feature extraction module includes: The business needs analysis submodule is used to analyze business needs and determine the key features and indicators required for customer profiling, including: Basic information: age, gender, region; Consumption behavior: purchase amount, purchase frequency, purchase category; Interactive behavior: pages viewed, clicks, comments; The data extraction submodule is used to extract the corresponding data from the bitmap data model and filter out customers that meet specific conditions through bit operations; The statistical analysis submodule is used to perform statistical analysis on the extracted data and calculate the average purchase amount and purchase frequency index of customers; The customer portrait building submodule is used to build a comprehensive portrait for each customer based on the statistical analysis results, and divide customers into different types, including high-value customers, active customers, and potential customers; The customer portrait storage submodule is used to store customer portraits in a bitmap data model for subsequent query and use.

[0012] Furthermore, the circle group division module includes: The segmentation rule definition submodule is used to define the segmentation rules of circles based on the data in the customer portrait. Customers whose consumption amount exceeds a certain value are high-value customers, and customers whose purchase frequency is less than a certain number of times are low-frequency buyers. The operation logic conversion submodule is used to convert the defined circle group division rules into bitmap operation logic, set the bit corresponding to the customer whose consumption amount exceeds a certain value to 1, and set the bit corresponding to the customer whose purchase frequency is less than a certain number to 0; The operation logic documentation submodule is used to document the bitmap operation logic for the subsequent application and maintenance of the circle group determination module; The circle group determination module includes: The division rule acquisition submodule is used to obtain the defined circle group division rules and bitmap operation logic; The bitmap operation submodule is used to determine the circle group to which each customer belongs based on their characteristics and indicators using bitmap operation logic, wherein the bitmap operation logic includes: Bitwise AND operation: used to identify customers who meet multiple conditions at the same time; Bitwise OR operation: used to determine customers who meet any of the conditions; Bitwise exclusive OR operation XOR: used to determine customers who meet one condition but not another condition at the same time; The circle group determination submodule is used to determine the circle group to which each customer belongs based on the bitmap operation result, and to screen out customer IDs that meet the conditions based on the bitmap operation result. These customer IDs constitute the circle group.

[0013] An efficient customer segmentation selection method driven by bitmap includes the following steps: Step 1: Collect customer-related data from multiple data sources and clean, transform and integrate them; Step 2: Store the integrated data in a bitmap and build a data model; Step 3: Extract key features and indicators based on the stored data to build a comprehensive profile for each customer; Step 4: Use the customer portrait constructed in step 3 to clarify the rules for dividing circles; Step 5: Using the defined group rules, determine the group to which each customer belongs by querying and calculating in the bitmap; Step 6: Store the circle group results in a bitmap for subsequent query and use.

[0014] Furthermore, step 1 specifically includes the following steps: Step 1.1: Identify and determine the types of customer-related data that need to be collected, including basic user information, consumption behavior data, and interactive behavior data; Step 1.2: Determine the data sources, including internal data sources and external data sources. Internal data sources include order systems, CRM systems, and website access logs. External data sources include social media data and third-party data services. Step 1.3: Select appropriate data collection tools and techniques for data collection based on the characteristics of the data source; Step 1.4: Clean the collected data, including removing duplicate data, filling missing values, and correcting erroneous data; Step 1.5: Convert the cleaned data, unify the data formats of different data sources into a standard format, and convert the data types into a format suitable for bitmap storage; Step 1.6: Integrate the transformed data into a unified data storage system; Step 1.7: Perform quality checks on the integrated data to ensure accuracy and consistency; Step 1.8: Categorize and label the data according to business requirements.

[0015] Furthermore, step 2 specifically includes the following steps: Step 2.1: Determine the bitmap storage structure, including selecting an appropriate bitmap implementation to optimize storage space and query performance; Step 2.2: Map customer-related data into a bitmap structure, assign a unique identifier to each customer, and map the customer's characteristics and indicators to different bits of the bitmap; Step 2.3: Build a data model based on business needs, including creating customer dimension tables and behavior fact tables to support complex queries and analysis; Step 2.4: compress the bitmap data to further save storage space and improve query efficiency; Step 2.5: According to the definition of the data model, index the bitmap data to quickly locate and query the data of a specific customer; Step 2.6: Store the constructed bitmap data model in a suitable data storage system.

[0016] Furthermore, step 3 specifically includes the following steps: Step 3.1: Analyze business needs and determine the key characteristics and indicators required for customer profiles, including: Basic information: age, gender, region; Consumption behavior: purchase amount, purchase frequency, purchase category; Interactive behavior: pages viewed, clicks, comments; Step 3.2: Extract the corresponding data from the bitmap data model and filter out customers that meet the specific conditions through bit operations; Step 3.3: Perform statistical analysis on the extracted data, including calculating the average purchase amount and purchase frequency index of customers; Step 3.4: Based on the statistical analysis results, build a comprehensive profile for each customer and classify them into different types, including high-value customers, active customers, and potential customers; Step 3.5: Store the customer portrait in a bitmap data model for subsequent query and use.

[0017] Furthermore, step 4 specifically includes the following steps: Step 4.1: Based on the data in the customer portrait, define the rules for dividing circles. Customers whose spending exceeds a certain value are high-value customers, and customers whose purchase frequency is less than a certain number of times are low-frequency buyers. Step 4.2: Convert the defined circle group division rules into bitmap operation logic, set the bit corresponding to customers whose consumption amount exceeds a certain value to 1, and set the bit corresponding to customers whose purchase frequency is less than a certain number to 0; Step 4.3: Document the bitmap operation logic for application in step 5 and subsequent maintenance.

[0018] Furthermore, step 5 specifically includes the following steps: Step 5.1: Obtain the circle group division rules and bitmap operation logic defined in step 4; Step 5.2: For each customer, use bitmap operation logic to determine the group to which the customer belongs based on the customer's characteristics and indicators. The bitmap operation logic includes: Bitwise AND operation (AND): used to identify customers who meet multiple conditions at the same time. For example, if the circle group rule is "aged 30-40 and has purchased product A", the bitmaps representing these two conditions can be bitwise ANDed to obtain a bitmap of customers who meet both conditions.

[0019] Bitwise OR operation (OR): used to determine customers who meet any one of the conditions. For example, if the circle group rule is "aged 30-40 or has purchased product A", the bitmaps representing these two conditions can be bitwise ORed to obtain a bitmap of customers who meet any one of the conditions.

[0020] Bitwise XOR (XOR): used to identify customers who meet one condition but not another. For example, if the circle rule is "aged 30-40 but have never purchased product A", the bitmaps representing these two conditions can be XORed to obtain a bitmap of customers who meet the conditions.

[0021] Step 5.3: Determine the circle group to which each customer belongs based on the bitmap operation result. For example, based on the bitmap operation result, select customer IDs that meet the conditions. These customer IDs constitute a specific circle group.

[0022] Because the present invention adopts the above technical means, it has the following beneficial effects: Data collection and processing: This method collects customer data from multiple data sources and cleans, transforms and integrates them to ensure data quality and consistency. This helps improve the accuracy of subsequent data analysis and customer segmentation.

[0023] Bitmap storage structure: Optimize storage space and query performance by determining the appropriate bitmap storage structure. The efficiency of the bitmap structure makes data storage more compact and reduces storage space requirements.

[0024] Customer portrait construction: Based on the stored data, key features and indicators are extracted to build a comprehensive portrait for each customer. This helps to understand customer behavior and preferences more deeply and provide a basis for precision marketing.

[0025] Definition of circle rules: Use customer portraits to clarify the rules for dividing circles, making the division of circles more refined and targeted.

[0026] Bitmap operation logic: Based on the defined grouping rules, the group to which each customer belongs is determined by querying and calculating in the bitmap. Bitmap operation logic includes bitwise AND operation, bitwise OR operation and bitwise XOR operation, which can efficiently handle complex customer segmentation requirements.

[0027] Performance advantages: Compared with traditional SQL queries and multi-table join methods, bitmap operations have significant performance advantages when processing large-scale data, including faster query execution time, lower CPU utilization, less memory usage, and reduced disk I / O overhead.

[0028] Business application: This method can be widely used in business scenarios such as precision marketing, customer segmentation, and product recommendations to improve the effectiveness of marketing activities and customer satisfaction.

[0029] Code simplicity: Bitmap operations simplify code implementation, making the code more concise and easier to maintain.

[0030] Scalability: The bitmap structure has good scalability and can adapt to the growing amount of data and changes in business needs.

[0031] In summary, this solution shows significant advantages in data processing efficiency, storage optimization, customer segmentation accuracy and business application flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a simplified flow chart of the present invention; Figure 2 A simplified diagram of the process of collecting customer-related data from multiple data sources and cleaning, transforming and integrating them; Figure 3A simplified diagram of the process of building a data model to store the integrated data in a bitmap; Figure 4 A simple flow chart for using customer portraits to clarify the rules for dividing circles; Figure 5 A simplified flow chart of the process of determining the circle group to which each customer belongs by querying and calculating in the bitmap in order to use the rules of the defined circle groups. DETAILED DESCRIPTION

[0033] The following is a detailed description of the embodiments of the present invention. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, the present invention may be modified or replaced by equivalents.

[0034] In addition, in order to better illustrate the present invention, numerous specific details are given in the following specific embodiments. It will be understood by those skilled in the art that the present invention can also be implemented without these specific details.

[0035] The following is an implementation example written based on the content of the document, and is expanded and supplemented with the detailed description in the document: An efficient customer segmentation selection method driven by bitmap includes the following steps: Step 1: Collect customer-related data from multiple data sources and clean, transform and integrate them; Step 2: Store the integrated data in a bitmap and build a data model; Step 3: Extract key features and indicators based on the stored data to build a comprehensive profile for each customer; Step 4: Use the customer portrait constructed in step 3 to clarify the rules for dividing circles; Step 5: Using the defined group rules, determine the group to which each customer belongs by querying and calculating in the bitmap; Step 6: Store the circle group results in a bitmap for subsequent query and use.

[0036] Furthermore, step 1 specifically includes the following steps: Step 1.1: Identify and determine the types of customer-related data that need to be collected, including basic user information, consumption behavior data, and interactive behavior data; Step 1.2: Determine the data sources, including internal data sources and external data sources. Internal data sources include order systems, CRM systems, and website access logs. External data sources include social media data and third-party data services. Step 1.3: Select appropriate data collection tools and techniques for data collection based on the characteristics of the data source; Step 1.4: Clean the collected data, including removing duplicate data, filling missing values, and correcting erroneous data; Step 1.5: Convert the cleaned data, unify the data formats of different data sources into a standard format, and convert the data types into a format suitable for bitmap storage; Step 1.6: Integrate the transformed data into a unified data storage system; Step 1.7: Perform quality checks on the integrated data to ensure accuracy and consistency; Step 1.8: Categorize and label the data according to business requirements.

[0037] Further, step 2: storing the integrated data in a bitmap and constructing a data model, specifically the following steps: Step 2 specifically includes the following steps: Step 2.1: Determine the bitmap storage structure, including selecting a suitable bitmap implementation to optimize storage space and query performance; In this step, we need to select a suitable bitmap implementation. Considering the optimization of storage space and query performance, we can choose a bitmap implementation that is both efficient in storage and fast in query. For example, Roaring Bitmaps is a commonly used bitmap implementation that is particularly effective when processing sparse data because it can dynamically choose to use arrays or bitmaps to store data, thereby saving space and improving query efficiency.

[0038] Step 2.2: Map customer-related data into a bitmap structure, assign a unique identifier to each customer, and map the customer's characteristics and indicators to different bits of the bitmap; to improve efficiency and scalability, we can use a hash function to map the customer identifier to the bits in the bitmap. At the same time, we need to ensure the consistency of the mapping so that each customer's data can be accurately located when querying.

[0039] Step 2.3: Build a data model based on business needs, including creating customer dimension tables and behavior fact tables to support complex queries and analysis; for example, you can create a customer dimension table to store basic customer information (such as age, gender, region, etc.), and a behavior fact table to store customer behavior data (such as purchase amount, purchase frequency, browsed pages, etc.).

[0040] Step 2.4: compress the bitmap data to further save storage space and improve query efficiency; various compression algorithms can be used as embodiments, such as Run-Length Encoding (RLE), Delta Encoding, etc. These algorithms can help us reduce redundant data in the bitmap, thereby improving storage efficiency and query performance.

[0041] Step 2.5: According to the definition of the data model, index the bitmap data to quickly locate and query the data of a specific customer; indexing can help us quickly locate and query the data of a specific customer. We can use index structures such as B-trees and hash indexes to achieve this goal. At the same time, we can also consider using bitmap indexes, which can further improve query efficiency, especially when dealing with large amounts of data.

[0042] Step 2.6: Store the constructed bitmap data model in a suitable data storage system. To support efficient query and analysis, we can choose to use a columnar storage system such as Apache HBase, Google Bigtable, etc. These systems can provide high-performance read and write operations and support the storage and processing of large-scale data.

[0043] Furthermore, step 3 extracts key features and indicators based on the stored data to build a comprehensive profile for each customer, which specifically includes the following steps: Step 3.1: Analyze business needs and determine the key characteristics and indicators required for customer profiles, including: Basic information: age, gender, region; Consumption behavior: purchase amount, purchase frequency, purchase category; Interactive behavior: pages viewed, clicks, comments; Step 3.2: Extract the corresponding data from the bitmap data model and filter out customers that meet the specific conditions through bit operations; Step 3.3: Perform statistical analysis on the extracted data, including calculating the average purchase amount and purchase frequency index of customers; Step 3.4: Based on the statistical analysis results, build a comprehensive profile for each customer and classify them into different types, including high-value customers, active customers, and potential customers; Step 3.5: Store the customer portrait in a bitmap data model for subsequent query and use.

[0044] As an example of step 3: 3a. Feature and indicator selection: Select key features and indicators based on business needs and the purpose of customer profiling. For example, if we want to build a customer profile for personalized recommendations, we may need to select features and indicators such as the customer's purchase history, browsing behavior, and preference settings.

[0045] 3b. Data extraction: Extract selected features and indicators from the stored data. This may involve operations such as querying databases, reading files, calling APIs, etc. We need to ensure that the extracted data is accurate and complete.

[0046] 3c. Feature and indicator calculation: Based on the extracted data, the selected features and indicators are calculated. This may involve statistical analysis, machine learning algorithms, data mining techniques, etc. For example, we can use clustering algorithms to calculate customers' purchase preferences, or use time series analysis to calculate customers' purchase frequency.

[0047] 3d. Customer portrait construction: Integrate the calculated features and indicators into the customer portrait. This may involve operations such as data visualization, report generation, and data storage. We need to ensure that the customer portrait can intuitively display the customer's features and indicators and can be easily queried and analyzed.

[0048] 3e. Customer profile update: Regularly update the customer profile to reflect the customer's latest characteristics and indicators. This may involve repeating steps 3a to 3d, or using incremental update techniques to reduce computational and storage overhead.

[0049] 3f. Customer profile validation: Verify the accuracy and effectiveness of the customer profile. This may involve comparing data on the customer profile with actual customer behavior, or using A / B testing to evaluate the impact of the customer profile on business goals.

[0050] Further, step 4: use the customer portrait constructed in step 3 to clarify the rules for dividing circles, which specifically includes the following steps: Step 4.1: Based on the data in the customer portrait, define the rules for dividing circles. Customers whose spending exceeds a certain value are high-value customers, and customers whose purchase frequency is less than a certain number of times are low-frequency buyers. Step 4.2: Convert the defined circle group division rules into bitmap operation logic, set the bit corresponding to customers whose consumption amount exceeds a certain value to 1, and set the bit corresponding to customers whose purchase frequency is less than a certain number to 0; Step 4.3: Document the bitmap operation logic for application in step 5 and subsequent maintenance.

[0051] As an example of step 4: Customer profile analysis: Analyze the customer profile constructed in step 3 to identify the key characteristics and indicators of the customer. This may involve statistical analysis, data mining techniques, machine learning algorithms, etc. For example, use clustering algorithms to identify customer groups with similar purchasing behaviors, or use association rule mining to discover customer purchasing patterns and preferences.

[0052] Definition of circle characteristics: Based on the results of customer profile analysis, define the characteristics and indicators of the circle. This involves selecting distinguishing characteristics and indicators, as well as determining the size and number of circles. For example, define a circle as "high-value customers", whose characteristics include high purchase frequency, large average order value, high customer satisfaction, etc.

[0053] Formulate group segmentation rules: Formulate group segmentation rules based on group feature definitions. This involves setting thresholds, conditions, weights, etc. For example, set a rule to classify customers whose purchase frequency is greater than a certain value and whose average order value is greater than another value as "high-value customers."

[0054] Segmentation rule verification: Verify the accuracy and effectiveness of segmentation rules. This may involve comparing segmentation results with actual customer behavior data, or using A / B testing to evaluate the impact of segmentation on business goals.

[0055] Application of circle group division rules: Apply circle group division rules to actual business scenarios, such as precision marketing, customer segmentation, product recommendations, etc. This may involve operations such as data integration, API calls, and system integration.

[0056] Optimize circle group division rules: Optimize circle group division rules based on circle group division results and business feedback. This can be done by adjusting thresholds, conditions, weights, etc. For example, if a circle group is found to be ineffective, you can try to adjust the division rules to better meet business needs.

[0057] Further, step 5: using the defined group rules, determine the group to which each customer belongs by querying and calculating in the bitmap, specifically including the following steps: Step 5.1: Obtain the circle group division rules and bitmap operation logic defined in step 4; Step 5.2: For each customer, use bitmap operation logic to determine the group to which the customer belongs based on the customer's characteristics and indicators. The bitmap operation logic includes: Bitwise AND operation (AND): used to identify customers who meet multiple conditions at the same time. For example, if the circle group rule is "aged 30-40 and has purchased product A", the bitmaps representing these two conditions can be bitwise ANDed to obtain a bitmap of customers who meet both conditions.

[0058] Bitwise OR operation (OR): used to determine customers who meet any one of the conditions. For example, if the circle group rule is "aged 30-40 or has purchased product A", the bitmaps representing these two conditions can be bitwise ORed to obtain a bitmap of customers who meet any one of the conditions.

[0059] Bitwise XOR (XOR): used to identify customers who meet one condition but not another. For example, if the circle rule is "aged 30-40 but have never purchased product A", the bitmaps representing these two conditions can be XORed to obtain a bitmap of customers who meet the conditions.

[0060] Step 5.3: Determine the circle group to which each customer belongs based on the bitmap operation result. For example, based on the bitmap operation result, select customer IDs that meet the conditions. These customer IDs constitute a specific circle group.

[0061] Example 1 As an example of the prior art: there is an e-commerce platform with a user table, a product table, and an order table. We want to find out the users who purchased both product A and product B, the users who purchased product A or product B, and the users who purchased product A but not product B.

[0062] MySQL JOIN implementation: 1: First, the table structure is as follows: User table (user_id, user_name) Product table (product_id, product_name) Order table (order_id, user_id, product_id) 2: To find users who purchased both product A and product B, you can use the following SQL: SELECT u.user_name FROM users u JOIN orders o1 ON u.user_id = o1.user_id AND o1.product_id = (SELECTproduct_id FROM products WHEREproduct_name = 'Product A') JOIN orders o2 ON u.user_id = o2.user_id AND o2.product_id = (SELECTproduct_id FROM products WHEREproduct_name = 'Product B'); Three: To find users who purchased product A or product B: SELECT u.user_name FROM users u JOIN orders o ON u.user_id = o.user_id WHERE o.product_id = (SELECTproduct_id FROM products WHEREproduct_name = 'Product A') OR o.product_id = (SELECT product_id FROM products WHERE product_name= 'Product B'); 4: To find users who purchased product A but not product B: SELECT u.user_name FROM users u JOIN orders o1 ON u.user_id = o1.user_id AND o1.product_id = (SELECTproduct_id FROM products WHEREproduct_name = 'Product A') WHERE u.user_id NOT IN( SELECT user_id FROM orders WHERE product_id = (SELECT product_id FROM products WHERE product_name = 'Product B') ); As an embodiment of the method of the present invention: 1. Use Bitmap to represent the user's purchase behavior. Each product has a corresponding Bitmap, where each bit represents a user. If the user purchases the product, the corresponding bit is 1, otherwise it is 0.

[0063] 2. To find users who have purchased both product A and product B, simply perform a bitwise AND operation on the Bitmaps of product A and product B: SELECT bitmap_count(bitmap_and(bitmap_column_for_product_A,bitmap_column_for_product_B)) AS num_users; 3. To find out the users who bought product A or product B, perform a bitwise OR operation: SELECT bitmap_count(bitmap_or(bitmap_column_for_product_A,bitmap_column_for_product_B)) AS num_users; 4. To find users who bought product A but not product B, perform bitwise AND and bitwise XOR operations: SELECT bitmap_count(bitmap_and(bitmap_column_for_product_A,bitmap_xor(bitmap_column_for_product_A, bitmap_column_for_product_B))) AS num_users; Performance index comparison: Query execution time: The Bitmap of the present invention: When processing large-scale data, the bit operation of Bitmap is extremely fast, and complex set operations can usually be completed in milliseconds or even microseconds. For example, for millions or even tens of millions of user and order data, Bitmap intersection and difference operations can produce results in a very short time.

[0064] MySQL JOIN: When the amount of data is large, MySQL's JOIN operation may require a lot of table scanning, index search and data comparison, and the execution time may be long. For the same amount of data, MySQL's query may take several seconds or even tens of seconds, especially when the table does not have a suitable index or the connection conditions are complex.

[0065] CPU Utilization: The Bitmap of the present invention: Since the Bitmap operation is mainly based on efficient bit operations, the CPU utilization is relatively low and stable. When performing intersection and difference operations, the CPU occupancy rate will not increase sharply like complex SQL queries, and the CPU resources can be more effectively utilized for parallel processing.

[0066] MySQL JOIN: JOIN operations may result in high CPU utilization, especially when dealing with complex joins and large amounts of data. If the join conditions involve multiple fields from multiple tables, or operations such as sorting and grouping are required, CPU resource consumption will be more significant, which may lead to a decrease in overall system performance.

[0067] Memory usage: The Bitmap of the present invention: The Bitmap data structure itself is relatively compact and occupies less memory space. Moreover, Bitmap is more efficient in memory management and optimization, and can process large-scale data under limited memory resources. Even for high-cardinality columns (such as user ID), the memory usage of Bitmap is relatively controllable.

[0068] MySQL JOIN: When performing a JOIN operation, MySQL needs to load the data of the tables involved in the connection into memory for comparison and processing. If the amount of data in the table is large, or a large number of intermediate result sets are generated during the connection process, a large amount of memory space may be occupied. In particular, for queries that have not been optimized, insufficient memory may occur.

[0069] Disk I / O overhead: The Bitmap of the present invention generally adopts column storage and efficient data compression technology, which can reduce the storage volume of data on the disk, thereby reducing the disk I / O overhead. In the Bitmap intersection and difference operation, the data is read and processed mainly in the memory, reducing the frequent access to the disk.

[0070] MySQL JOIN: MySQL's table storage method may cause data to be stored on disk in an incompact manner. When performing JOIN operations, it may be necessary to frequently read data blocks from disk. Especially when the table has a large amount of data and there is no suitable index, the disk I / O overhead will increase significantly, affecting query performance.

[0071] For example, in an actual e-commerce platform, if there are millions or even tens of millions of users and products, it may take several minutes or even longer to complete the query using MySQL's JOIN operation, while using Bitmap operations can get results in seconds. In addition, as the amount of data continues to grow, MySQL's performance may further decline, while Bitmap can still maintain good performance.

[0072] In summary, when dealing with complex set operations, Bitmap's implementation of intersection, union and difference operations has obvious advantages over traditional MySQL's JOIN, especially in terms of performance, storage space and code simplicity.

Claims

1. An efficient customer segmentation and selection device driven by a bitmap, comprising: Data collection module, used to collect customer-related data from multiple data sources and clean, transform and integrate them; A bitmap storage module is used to store the integrated data in a bitmap and construct a data model; Feature extraction module, which is used to extract key features and indicators based on stored data to build a comprehensive profile for each customer; The circle group division module is used to use customer portraits to clarify the rules for dividing circles; A circle group determination module is used to determine the circle group to which each customer belongs by querying and calculating in the bitmap using defined circle group rules; The result storage module is used to store the circle group results in a bitmap for subsequent query and use.

2. According to the bitmap-driven efficient customer segmentation and selection device of claim 1, the data collection module comprises: The data type identification submodule is used to identify and determine the types of customer-related data that need to be collected, including basic user information, consumption behavior data, and interactive behavior data; The data source determination submodule is used to determine the data source, including internal data sources and external data sources. The internal data sources include the order system, CRM system, and website access logs, and the external data sources include social media data and third-party data services. The data collection submodule is used to select appropriate data collection tools and technologies for data collection according to the characteristics of the data source; The data cleaning submodule is used to clean the collected data, including removing duplicate data, filling missing values, and correcting erroneous data; The data conversion submodule is used to convert the cleaned data, unify the data formats of different data sources into a standard format, and convert the data type into a format suitable for bitmap storage; A data integration submodule is used to integrate the converted data into a unified data storage system; The quality check submodule is used to perform quality checks on the integrated data to ensure the accuracy and consistency of the data; The data classification and labeling submodule is used to classify and label data according to business needs.

3. The efficient customer segmentation and selection device using bitmap drive according to claim 2, wherein the bitmap storage module comprises: A bitmap structure determination submodule is used to determine the bitmap storage structure, including selecting a suitable bitmap implementation method to optimize storage space and query performance; A data mapping submodule, which is used to map customer-related data into a bitmap structure, assign a unique identifier to each customer, and map the customer's characteristics and indicators to different bits of the bitmap; The data model building submodule is used to build data models according to business requirements, including creating customer dimension tables and behavior fact tables; A bitmap data compression submodule is used to compress bitmap data to save storage space and improve query efficiency; The data indexing submodule is used to index the bitmap data according to the definition of the data model so as to quickly locate and query the data of a specific customer; The data storage submodule is used to store the constructed bitmap data model in a suitable data storage system.

4. The efficient customer segmentation and selection device using bitmap drive according to claim 3, The feature extraction module includes: The business needs analysis submodule is used to analyze business needs and determine the key features and indicators required for customer profiling, including: Basic information: age, gender, region; Consumption behavior: purchase amount, purchase frequency, purchase category; Interactive behavior: pages viewed, clicks, comments; The data extraction submodule is used to extract the corresponding data from the bitmap data model and filter out customers that meet specific conditions through bit operations; The statistical analysis submodule is used to perform statistical analysis on the extracted data and calculate the average purchase amount and purchase frequency index of customers; The customer portrait building submodule is used to build a comprehensive portrait for each customer based on the statistical analysis results, and divide customers into different types, including high-value customers, active customers, and potential customers; The customer portrait storage submodule is used to store customer portraits in a bitmap data model for subsequent query and use.

5. According to the bitmap-driven efficient customer segmentation and selection device of claim 4, the group division module comprises: The segmentation rule definition submodule is used to define the segmentation rules of circles based on the data in the customer portrait. Customers whose consumption amount exceeds a certain value are high-value customers, and customers whose purchase frequency is less than a certain number of times are low-frequency buyers. The operation logic conversion submodule is used to convert the defined circle group division rules into bitmap operation logic, set the bit corresponding to the customer whose consumption amount exceeds a certain value to 1, and set the bit corresponding to the customer whose purchase frequency is less than a certain number to 0; The operation logic documentation submodule is used to document the bitmap operation logic for the subsequent application and maintenance of the circle group determination module; The circle group determination module includes: The division rule acquisition submodule is used to obtain the defined circle group division rules and bitmap operation logic; The bitmap operation submodule is used to determine the circle group to which each customer belongs based on their characteristics and indicators using bitmap operation logic, wherein the bitmap operation logic includes: Bitwise AND operation: used to identify customers who meet multiple conditions at the same time; Bitwise OR operation: used to determine customers who meet any of the conditions; Bitwise exclusive OR operation XOR: used to determine customers who meet one condition but not another condition at the same time; The circle group determination submodule is used to determine the circle group to which each customer belongs based on the bitmap operation result, and to screen out customer IDs that meet the conditions based on the bitmap operation result. These customer IDs constitute the circle group.

6. An efficient customer segmentation selection method using bitmap drive, characterized in that: The following steps are involved: Step 1: Collect customer-related data from multiple data sources and clean, transform and integrate them; Step 2: Store the integrated data in a bitmap and build a data model; Step 3: Extract key features and indicators based on the stored data to build a comprehensive profile for each customer; Step 4: Use the customer portrait constructed in step 3 to clarify the rules for dividing circles; Step 5: Using the defined group rules, determine the group to which each customer belongs by querying and calculating in the bitmap; Step 6: Store the circle group results in a bitmap for subsequent query and use.

7. The efficient customer segmentation selection method using bitmap drive according to claim 6, characterized in that: The following steps are involved: Step 1 specifically includes the following steps: Step 1.1: Identify and determine the types of customer-related data that need to be collected, including basic user information, consumption behavior data, and interactive behavior data; Step 1.2: Determine the data sources, including internal data sources and external data sources. Internal data sources include order systems, CRM systems, and website access logs. External data sources include social media data and third-party data services. Step 1.3: Select appropriate data collection tools and techniques for data collection based on the characteristics of the data source; Step 1.4: Clean the collected data, including removing duplicate data, filling missing values, and correcting erroneous data; Step 1.5: Convert the cleaned data, unify the data formats of different data sources into a standard format, and convert the data types into a format suitable for bitmap storage; Step 1.6: Integrate the transformed data into a unified data storage system; Step 1.7: Perform quality checks on the integrated data to ensure accuracy and consistency; Step 1.8: Categorize and label the data according to business requirements.

8. The efficient customer segmentation selection method using bitmap drive according to claim 7 is characterized in that: The following steps are involved: Step 2 specifically includes the following steps: Step 2.1: Determine the bitmap storage structure, including selecting an appropriate bitmap implementation to optimize storage space and query performance; Step 2.2: Map customer-related data into a bitmap structure, assign a unique identifier to each customer, and map the customer's characteristics and indicators to different bits of the bitmap; Step 2.3: Build a data model based on business needs, including creating customer dimension tables and behavior fact tables to support complex queries and analysis; Step 2.4: compress the bitmap data to save storage space and improve query efficiency; Step 2.5: According to the definition of the data model, index the bitmap data to quickly locate and query the data of a specific customer; Step 2.6: Store the constructed bitmap data model in a suitable data storage system.

9. The efficient customer segmentation selection method using bitmap drive according to claim 8, characterized in that: The following steps are involved: Step 3 specifically includes the following steps: Step 3.1: Analyze business needs and determine the key characteristics and indicators required for customer profiles, including: Basic information: age, gender, region; Consumption behavior: purchase amount, purchase frequency, purchase category; Interactive behavior: pages viewed, clicks, comments; Step 3.2: Extract the corresponding data from the bitmap data model and filter out customers that meet the specific conditions through bit operations; Step 3.3: Perform statistical analysis on the extracted data, including calculating the average purchase amount and purchase frequency index of customers; Step 3.4: Based on the statistical analysis results, build a comprehensive profile for each customer and classify them into different types, including high-value customers, active customers, and potential customers; Step 3.5: Store the customer portrait in a bitmap data model for subsequent query and use.

10. The efficient customer segmentation selection method using bitmap drive according to claim 9, characterized in that: The following steps are involved: Step 4 specifically includes the following steps: Step 4.1: Based on the data in the customer portrait, define the rules for dividing circles. Customers whose spending exceeds a certain value are high-value customers, and customers whose purchase frequency is less than a certain number of times are low-frequency buyers. Step 4.2: Convert the defined circle group division rules into bitmap operation logic, set the bit corresponding to customers whose consumption amount exceeds a certain value to 1, and set the bit corresponding to customers whose purchase frequency is less than a certain number to 0; Step 4.3: Document the bitmap operation logic for application in step 5 and subsequent maintenance; Step 5 specifically includes the following steps: Step 5.1: Obtain the circle group division rules and bitmap operation logic defined in step 4; Step 5.2: For each customer, use bitmap operation logic to determine the group to which the customer belongs based on the customer's characteristics and indicators. The bitmap operation logic includes: Bitwise AND operation: used to identify customers who meet multiple conditions at the same time; Bitwise OR operation: used to determine customers who meet any of the conditions; Bitwise exclusive OR operation XOR: used to determine customers who meet one condition but not another condition at the same time; Step 5.3: According to the bitmap operation result, determine the circle group to which each customer belongs, and according to the bitmap operation result, filter out the customer IDs that meet the conditions, and these customer IDs constitute the circle group.

Images