Real-time marketing recommendation method and device and medium

Abstract

The invention provides a real-time marketing recommendation method and device and a medium, and relates to the technical field of information processing, and the method comprises the steps: obtaining marketing recommendation demands customized by a first user in real time, the marketing recommendation demands comprising a marketing recommendation source data demand, a marketing recommendation data content demand and a marketing recommendation data subscription demand; selecting and combining a pre-designed first calculation logic module according to a marketing recommendation source data demand, and collecting marketing recommendation source data according to the first calculation logic module; selecting and combining a pre-designed second calculation logic module according to the marketing recommendation data content requirement, and analyzing the marketing recommendation source data by using the second calculation logic module to generate real-time marketing recommendation data with specified content; and selecting and combining a pre-designed third calculation logic module according to a marketing recommendation data subscription demand, and pushing the real-time marketing recommendation data to a subscribed marketing recommendation terminal by using the third calculation logic module. According to the invention, marketing, maintenance and recommendation integrated data service is realized.

Description

Technical Field

[0001] The present disclosure relates at least to the field of information processing technologies, and particularly to a real-time marketing recommendation method, apparatus, and medium. Background Art

[0002] With the rise of big data and the update and iteration of real-time computing frameworks, the traditional offline data storage and computing solution based on the Hadoop (a distributed system infrastructure) ecosystem can no longer meet the user's demands for highly time-sensitive data. In particular, as the company's business scenario operations continue to deepen, the demand for real-time data empowerment becomes more intense. The traditional data empowerment model cannot meet the user experience in the marketing scenario, and at the same time brings challenges to the agile development ability and data reusability of real-time data. Summary of the Invention

[0003] The technical problem to be solved by the present disclosure is to provide a real-time marketing recommendation method, apparatus, and medium to solve the problem of how to improve the real-time data empowerment in the marketing scenario in view of the above deficiencies.

[0004] In a first aspect, the present disclosure provides a real-time marketing recommendation method, the method comprising:

[0005] Obtaining the marketing recommendation requirements customized in real time by a first user, where the marketing recommendation requirements include marketing recommendation source data requirements, marketing recommendation data content requirements, and marketing recommendation data subscription requirements;

[0006] Selecting and combining a pre-designed first computing logic module according to the marketing recommendation source data requirements, and collecting marketing recommendation source data according to the first computing logic module;

[0007] Selecting and combining a pre-designed second computing logic module according to the marketing recommendation data content requirements, and analyzing the marketing recommendation source data using the second computing logic module to generate real-time marketing recommendation data with specified content;

[0008] Selecting and combining a pre-designed third computing logic module according to the marketing recommendation data subscription requirements, and pushing the real-time marketing recommendation data to the subscribed marketing recommendation terminal using the third computing logic module.

[0009] Further, the method further comprises:

[0010] Receiving user-defined functions (UDFs) developed by a second user according to the marketing recommendation source data requirements, marketing recommendation data content requirements, and marketing recommendation data subscription requirements respectively;

[0011] Generating a first computing logic module, a second computing logic module, and a third computing logic module according to the UDFs respectively.

[0012] Further, according to the marketing recommendation source data requirements, select and combine pre-designed first calculation logic modules, and collect marketing recommendation source data according to the first calculation logic modules, specifically including:

[0013] Obtain several real-time data sources that need to collect marketing recommendation source data and offline data sources located in the real-time data warehouse according to the marketing recommendation source data requirements, and determine the data collection rules for each real-time data source and the data update period for the offline data sources;

[0014] Receive multiple first calculation logic modules specified by the third user through the visual interface and the connection relationships of each first calculation logic module. The first calculation logic module includes several fourth calculation logic modules that respectively collect data from several real-time data sources according to the corresponding data collection rules, a fifth calculation logic module that updates data to the offline data source according to the data update period, and a sixth calculation module that collects data from the offline data source;

[0015] Generate a Flink SQL task according to multiple first calculation logic modules and their connection relationships, and collect marketing recommendation source data from the data sources according to the Flink SQL task, including collecting streaming data from several real-time data sources and batch data from the offline data source. Among them, Flink is a unified processing model for streaming data and batch data, and SQL is a structured query language.

[0016] Further, collect marketing recommendation source data from the data sources according to the Flink SQL task, specifically including:

[0017] Dynamically apply for a TaskManager according to the parallelism and resource requirements of the current Flink SQL task, and create a running container for each TaskManager;

[0018] Store the status data of the Flink job in the remote distributed file system DFS, and multiple TaskManagers share the status data through DFS;

[0019] If a fault occurs in the Flink job of a certain container, the certain container recovers the status data from DFS through status lazy loading and delayed pruning.

[0020] Further, according to the marketing recommendation data content requirements, select and combine pre-designed second calculation logic modules, and use the second calculation logic modules to analyze the marketing recommendation source data to generate real-time marketing recommendation data of specified content, specifically including:

[0021] Obtain the specified content that needs to generate real-time marketing recommendation data according to the marketing recommendation data content requirements. The specified content includes user portraits, product portraits, the real-time popularity of products, and the degree of user interest in products;

[0022] Determine a real-time task for data analysis of marketing recommendation source data according to the specified content. The real-time task includes: user profiling task, product profiling task, product popularity list task, and user-product collaborative filtering task;

[0023] Select the second calculation logic module corresponding to each real-time task, and input the marketing recommendation source data into each second calculation logic module to obtain real-time marketing recommendation data including user profiles, product profiles, product popularity lists, and user-product relationship pairs where the user's interest in the product is greater than the threshold.

[0024] Furthermore, the method further includes:

[0025] Take the real-time marketing recommendation data as a real-time data asset, and record the data lineage relationship between the real-time marketing recommendation data and the marketing recommendation source data to establish a real-time data asset catalog;

[0026] The fourth user manages the real-time data asset catalog, including: managing the same real-time marketing recommendation data requirements and marketing recommendation source data requirements of multiple first users, and promoting hot real-time marketing recommendation data according to the heat map of using real-time marketing recommendation data.

[0027] Furthermore, according to the marketing recommendation data subscription requirements, select and combine pre-designed third calculation logic modules, and use the third calculation logic modules to push the real-time marketing recommendation data to the subscribed marketing recommendation terminals. Specifically, it includes:

[0028] Obtain the subscribed marketing recommendation terminals, the data push methods subscribed by each marketing recommendation terminal, and the content of the real-time marketing recommendation data according to the marketing recommendation data subscription requirements;

[0029] Select the third calculation logic module with the corresponding data push method, and use the third calculation logic module to send the corresponding real-time marketing recommendation data content to each marketing recommendation terminal. The third calculation logic module is implemented based on Flink, and Flink is a unified processing model for stream data and batch data.

[0030] Furthermore, using the third calculation logic module to send the corresponding real-time marketing recommendation data content to each marketing recommendation terminal specifically includes:

[0031] Use the seventh calculation logic module to send real-time marketing recommendation data including user profiles, product profiles, product popularity lists, and user-product relationship pairs where the user's interest in the product is greater than the threshold to the marketing recommendation terminal of the first user;

[0032] Use the eighth computing logic module to send real-time marketing recommendation data including the popular products in the product popularity list and the products with the fifth user's interest level greater than the threshold to the marketing recommendation terminal of the fifth user, where the fifth user is a customer of the first user.

[0033] In a second aspect, the present disclosure provides a real-time marketing recommendation device, the device comprising:

[0034] A requirements unit, configured to obtain the marketing recommendation requirements customized by the first user in real time, where the marketing recommendation requirements include marketing recommendation source data requirements, marketing recommendation data content requirements, and marketing recommendation data subscription requirements;

[0035] An acquisition unit, connected to the requirements unit, configured to select and combine a pre-designed first computing logic module according to the marketing recommendation source data requirements, and acquire marketing recommendation source data according to the first computing logic module;

[0036] A generation unit, connected to the acquisition unit, configured to select and combine a pre-designed second computing logic module according to the marketing recommendation data content requirements, and analyze the marketing recommendation source data using the second computing logic module to generate real-time marketing recommendation data of specified content;

[0037] A push unit, connected to the generation unit, configured to select and combine a pre-designed third computing logic module according to the marketing recommendation data subscription requirements, and push the real-time marketing recommendation data to the subscribed marketing recommendation terminal using the third computing logic module.

[0038] In a third aspect, the present disclosure provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is run by a processor, the real-time marketing recommendation method described above is implemented.

[0039] The present disclosure provides a real-time marketing recommendation method, device, and medium. By analyzing the marketing recommendation requirements customized by the first user in real time, selecting and combining the computing logic modules that meet the user's marketing recommendation requirements, an integrated marketing maintenance recommendation data service that collects source data, generates marketing recommendation data, and pushes marketing recommendation content to meet the user's real-time needs is realized, low-code development and flexible marketing recommendation capability combination are achieved, and real-time data empowerment for marketing scenarios is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of a real-time marketing recommendation method according to an embodiment of the present disclosure;

[0041] Figure 2 is a schematic structural diagram of a real-time marketing recommendation device according to an embodiment of the present disclosure;

[0042] Figure 3It is the overall architecture diagram of a real-time marketing recommendation device according to an embodiment of the present disclosure;

[0043] Figure 4 It is the overall architecture diagram of another real-time marketing recommendation device according to an embodiment of the present disclosure;

[0044] Figure 5 It is the architecture diagram of a real-time task configuration method according to an embodiment of the present disclosure;

[0045] Figure 6 It is the architecture diagram of a data bus implementation method according to an embodiment of the present disclosure;

[0046] Figure 7 It is the architecture diagram of a memory-computation separation method according to an embodiment of the present disclosure;

[0047] Figure 8 It is the comparison schematic diagram of a job recovery method according to an embodiment of the present disclosure;

[0048] Figure 9 It is the architecture diagram of a real-time data asset catalog method according to an embodiment of the present disclosure. Detailed implementation manners

[0049] To enable those skilled in the art to better understand the technical solutions of the present disclosure, the following will further describe the embodiments of the present disclosure in detail with reference to the accompanying drawings.

[0050] It can be understood that the specific embodiments and the accompanying drawings described herein are only used to explain the present disclosure, rather than limiting the present disclosure.

[0051] It can be understood that, without conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other.

[0052] It can be understood that, for the convenience of description, only the parts related to the present disclosure are shown in the accompanying drawings of the present disclosure, and the parts unrelated to the present disclosure are not shown in the accompanying drawings.

[0053] It can be understood that each module and unit involved in the embodiments of the present disclosure may correspond to only one entity structure, or may be composed of multiple entity structures, or multiple modules and units may also be integrated into one entity structure.

[0054] It can be understood that, without conflict, the functions and steps marked in the flowcharts and block diagrams of the present disclosure may occur in an order different from that marked in the accompanying drawings.

[0055] It is understandable that in the flowcharts and block diagrams of the present disclosure, the possible architectures, functions, and operations of systems, devices, equipment, and methods according to various embodiments of the present disclosure are shown. Among them, each block in the flowchart or block diagram may represent a module, unit, program segment, or code, which contains executable instructions for implementing the specified function. Moreover, each block or combination of blocks in the block diagram and flowchart can be implemented by a hardware-based device for implementing the specified function, or by a combination of hardware and computer instructions.

[0056] It is understandable that the modules and units involved in the embodiments of the present disclosure can be implemented in software or in hardware. For example, the modules and units can be located in the processor.

[0057] Embodiment 1:

[0058] As Figure 1 shown, the present disclosure provides a real-time marketing recommendation method, and the method includes:

[0059] S1. Obtain the marketing recommendation requirements customized by the first user in real time. The marketing recommendation requirements include marketing recommendation source data requirements, marketing recommendation data content requirements, and marketing recommendation data subscription requirements;

[0060] S2. According to the marketing recommendation source data requirements, select and combine the pre-designed first calculation logic module, and collect the marketing recommendation source data according to the first calculation logic module;

[0061] S3. According to the marketing recommendation data content requirements, select and combine the pre-designed second calculation logic module, and use the second calculation logic module to analyze the marketing recommendation source data to generate real-time marketing recommendation data of specified content;

[0062] S4. According to the marketing recommendation data subscription requirements, select and combine the pre-designed third calculation logic module, and use the third calculation logic module to push the real-time marketing recommendation data to the subscribed marketing recommendation terminal.

[0063] In this embodiment, the method analyzes the marketing recommendation requirements customized by the first user in real time, selects and combines the calculation logic modules that meet the user's marketing recommendation requirements, so as to realize an integrated marketing maintenance recommendation data service that integrates collecting source data, generating marketing recommendation data, and pushing marketing recommendation content to meet the user's real-time needs, realizes low-code development and flexible combination of marketing recommendation capabilities, and improves real-time data empowerment for marketing scenarios. As Figure 1 shown, the method corresponds to being applied to the device as Figure 2 shown.

[0064] Specifically, this embodiment provides a marketing recommendation solution based on real-time processing technology. Focusing on the fact that more and more data application scenarios have shifted from offline to real-time, and the existing real-time data warehouse architecture has very high maintenance costs for the entire data collection, processing, and usage link, it endeavors to improve the real-time data governance ability, promote the platformization of the real-time data warehouse construction, lower the threshold of real-time data development, improve the means of real-time data quality control, and enhance the real-time data asset service ability.

[0065] More specifically, when real-time data empowers marketing scenarios, the following problems exist: The traditional chimney-style data collection mode has high maintenance costs. There are many current database components, and personalized data collection requires code development, with low support efficiency and inconvenient centralized operation and maintenance; The response efficiency of real-time program development is low and the iteration cost is high. Developers need to have certain code capabilities, and the task release process takes a long time; The reusability of real-time data assets is low, lacking the ability to uniformly manage real-time data, which easily leads to situations such as duplicate construction of similar data and inconsistent calculation calibers of the same-name indicators, resulting in problems such as resource waste and inefficient data empowerment; The timeliness of traditional data recommendation models is poor. Traditional recommendation models mostly rely on offline data and lack flexibility, affecting the customer's usage perception.

[0066] In view of this, this embodiment proposes an intelligent recommendation solution based on real-time processing technology as Figures 3 - 4 shown. The solution architecture mainly includes four parts: a data transmission bus, real-time development, data asset services, and a recommendation system, realizing integrated data services of real-time data collection, visual data development, intelligent data real-time services, and marketing retention recommendations.

[0067] As Figure 3 shown, the solution includes: realizing the collection of multi-source heterogeneous source data based on the data bus, managing multi-source data based on the real-time data warehouse to form data assets, managing the data assets through registration, subscription, etc., and providing them to the marketing recommendation system through the data bus to realize customized marketing recommendation services.

[0068] As Figure 4As shown in the figure, the solution includes the following steps: ① Business personnel (the first user, who may be the marketing staff of the operator) configure marketing recommendation rules in combination with marketing recommendation strategies (thereby obtaining marketing recommendation requirements); ② Data collection personnel synchronize the real-time source data required by the marketing recommendation rules to the real-time data warehouse through the data bus capability; ③ Parse the corresponding marketing rules (marketing recommendation rules) to generate real-time tasks (generate a real-time task for each required marketing recommendation data content); ④ Write the real-time calculation results (real-time marketing recommendation data) into the storage component; ⑤ Publish the data results (real-time marketing recommendation data) to the real-time data (real-time assets, real-time data assets) directory; ⑥ Combine the real-time data asset directory subscription relationship (source and target information), and synchronize it to the application contact point through the data bus capability.

[0069] In one embodiment, the method further includes:

[0070] Receiving user-defined functions UDF developed by the second user according to the marketing recommendation source data requirements, marketing recommendation data content requirements, and marketing recommendation data subscription requirements respectively;

[0071] Generating a first calculation logic module, a second calculation logic module, and a third calculation logic module according to the UDF respectively.

[0072] In this embodiment, as Figure 5 shown, in the real-time task configuration function ( Figure 3 the task development function of the real-time data warehouse), it also includes a custom UDF function function to realize the custom configuration of complex calculation logic functions. This process can be developed by backend technical personnel (the second user). The backend technical personnel can confirm whether a new calculation function module (calculation logic module) needs to be developed according to the requirements received from the first user. If there is a corresponding calculation logic module, it can be directly called. This development process is not limited to being executed every time and can be realized through the custom development mode as Figure 3 shown. The real-time data warehouse supports uploading a jar (Java Archive) software compression package to configure the UDF function (User-Defined Function), enabling the backend technical personnel to accurately match the rules and enrich the calculation logic.

[0073] In one embodiment, S2. According to the marketing recommendation source data requirements, select and combine the pre-designed first calculation logic module, and collect the marketing recommendation source data according to the first calculation logic module. Specifically, it includes:

[0074] Obtain several real-time data sources that need to collect marketing recommendation source data and offline data sources located in the real-time data warehouse according to the marketing recommendation source data requirements, and determine the data collection rules for each real-time data source and the data update cycle for the offline data source;

[0075] Receiving multiple first computing logic modules specified by a third user through a visualization interface and the connection relationships of each first computing logic module. The first computing logic module includes several fourth computing logic modules that respectively collect data from several real-time data sources according to corresponding data collection rules, a fifth computing logic module that updates data to an offline data source according to a data update period, and a sixth computing module that collects data from the offline data source;

[0076] Generating a Flink SQL task according to the multiple first computing logic modules and their connection relationships, and collecting marketing recommendation source data from data sources according to the Flink SQL task, including collecting streaming data from several real-time data sources and collecting batch data from an offline data source. Among them, Flink is a unified processing model for streaming data and batch data, and SQL is a structured query language.

[0077] In this embodiment, as Figure 3 shown, the visualization development function can be used by front-end operation personnel (the third user) to configure the first computing logic module. As Figure 5 shown, it includes combining computing logic modules through drag-and-drop operations and identifying and escaping standard Flink sql tasks based on rules to achieve the collection of various source data, including real-time stream KafKa, calculation result Hudi, etc. As Figure 6 shown, the real-time platform ( Figure 4 the real-time tasks in the real-time data warehouse in it) can obtain historical data in the offline data domain through an indication diagram and obtain streaming data in the streaming data domain through streaming data. The offline data domain realizes the update of historical data by obtaining incremental data from the source data domain. All of these can be designed according to the needs of the first user for source data.

[0078] For example, around scenarios such as product recommendation for the online development of an app (application), and home business recommendation for community resident marketing, as Figure 3As shown in the figure, various types of source and target data are connected based on the bus configuration (mainly involving the connection of IP (Internet Protocol), ports, character sets, account information, etc.), realizing the subscription and collection of data sources and the writing of data to the target end (mainly including data such as user service subscriptions, user Internet access information, and location signaling), and being compatible with various application data storage components such as mysql (a relational database management system), Hudi (an open-source data lake framework), and Kafka (an open-source stream processing platform); real-time data development includes content such as visual development, data auditing, and operation and maintenance monitoring. Among them, the computing module uses Flink as a streaming processing component to perform operations such as data cleaning and association according to business requirements. Low-code visual development supports SQL syntax checking functions and custom UDF function configuration, and supports rapid iteration of requirements through agile development.

[0079] Combined with the scenario requirements such as product recommendations for app online development and home business recommendations for community station marketing, the data bus technology is mainly responsible for data transmission between different systems (modules), solves the compatibility problems of more than 20 data components such as MySQL, ob (a distributed database, OceanBase), and kafka, ensures the compatibility and adaptation of different types of data between the source end and the target end, and creates data transmission tasks (setting the synchronization period, divided into real-time / specified time); provides data supply for the real-time development platform. As Figure 6 As shown in the figure, offline data processing supports the storage and management of the main data set (immutable data set) and the pre-batch processed views (which are the results of offline data development precipitation. Views can be directly reused to avoid unnecessary redundant storage. A view is a virtual table that is defined through a query statement based on one or more actual tables (base tables) in the database. The view itself does not store data, and its data is dynamically obtained from the base table), filters out invalid information, effectively solves the needs of user differences, reduces the downstream data transmission, storage, and computing costs, and can also achieve data accuracy through offline storage of historical data (for example, real-time data is easily affected by network or other factors resulting in data loss, and it can be repaired and corrected through offline storage of historical data), and has high fault tolerance and fast repair capabilities. Real-time data (streaming data) processing realizes the real-time acquisition of source data, provides a real-time view of the latest data for the data users of the real-time platform (for scenarios that require multi-table association and have many table fields, only the required partial fields can be selected through the view to reduce data transmission consumption expenses), and ensures minimal latency. When it is necessary to roll back the increment, based on the marker bit adjustment, only the increment data of a certain version needs to be marked as invalid, and the merge logic is triggered to overwrite the old data. When the operation system modifies the data, an increment data of a higher version is inserted, and the merge logic is triggered to overwrite the old data.

[0080] Real-time development platform ( Figure 3For task development in [context], Flink is selected as the real-time computing framework. Developers can select the required real-time data sources on the visualization interface and connect and combine computing logic modules (the first computing logic module) through drag-and-drop to achieve the construction of real-time marketing rule modeling job tasks. In the task construction phase, the platform mainly has the following functions: For real-time data source configuration, the platform supports both the original streaming data constructed by DTS (Data Transmission Service, used to capture the change logs of databases) + Kakfa (message middleware, responsible for storing and distributing messages), and the micro-batch / real-time stream intermediate calculation result data sets processed by Hudi (Hadoop UpsertDelete Insert, a data storage framework built on top of the Hadoop ecosystem); the computing logic module will automatically convert the configured rules into Flink SQL tasks, which reduces the development threshold for front-end operation personnel and improves the flexibility of rule strategy adjustment; it supports uploading jar software compressed packages to configure UDF functions, enabling back-end technical personnel to accurately match rules and enrich the computing logic.

[0081] In one implementation, the marketing recommendation source data is collected from the data source according to the Flink SQL task, specifically including:

[0082] According to the parallelism and resource requirements of the current Flink SQL task, dynamically apply for TaskManager of the task manager, and create a running container for each TaskManager;

[0083] Store the status data of the Flink job in the remote distributed file system DFS, and multiple TaskManagers share the status data through DFS;

[0084] If a fault occurs in the Flink job of a certain container, the certain container recovers the status data from DFS through status lazy loading and delayed pruning.

[0085] In this embodiment, as Figure 7As shown in the figure, in the task deployment phase, the platform introduces the cloud-native Flink (Flink on Kubernetes) with a separated storage and computing architecture. Kubernetes, abbreviated as K8s, is an open-source system for automatically deploying, scaling, and managing containerized applications, enabling independent scaling of computing and storage resources, enhancing the flexibility of resource management, and shortening the job recovery time. In terms of resource management, first, Kubernetes (a container orchestration engine that can create multiple containers, each running an application instance, and manages this group of application instances through built-in load balancing strategies) provides powerful container orchestration capabilities. Users do not need to specify the number of TaskManagers (task managers, the working processes in the Flink cluster responsible for executing specific tasks) in advance but can dynamically apply for TaskManager resources according to the parallelism and resource requirements of the tasks, improving resource utilization. Second, in the separated storage and computing architecture, state data (the state information during the operation of the Flink job. When relying on previous inputs for calculation, a state table is needed to temporarily store intermediate data) can be stored in a remote distributed file system (DFS, Distributed File System) instead of the local disk. Therefore, multiple operators (referring to each operation unit in the Flink stream processing job) under the same task can share the same state file instead of each maintaining an independent state copy, reducing the redundancy of state data. In addition, once a Flink job fails, the system will directly load the latest state snapshot file from the DFS and then continue to read data from the correct position of the input stream according to the recorded offset and restart the job. Therefore, the Checkpoint (a technology for implementing a fault tolerance mechanism that ensures the ability to recover from the latest state in case of a failure by periodically saving state snapshots of the stream processing job, thus guaranteeing the accuracy and consistency of data processing) operation will also be faster.

[0086] As Figure 8As shown, two core strategies, namely state lazy loading and delayed pruning, are introduced in the storage-computation separation architecture, aiming to improve the efficiency of Flink jobs in recovering from failure states and shorten the job recovery duration. The state lazy loading strategy uses background processes to download state file data step by step and asynchronously, greatly reducing the I / O (Input / Output) pressure in the initial stage of job recovery and making the task recovery to the full-speed running performance stage smoother. The delayed pruning strategy also avoids the instantaneous large-volume data file downloads and actual merging overheads. By only loading metadata (also known as intermediate data, relay data, which is data about data, mainly information describing data properties and used to support functions such as indicating storage locations, historical data, resource lookup, file records, etc.) and constructing an LSM-tree structure (Log Structured MergeTree, a hierarchical and ordered data structure), it can start the service quickly and preferentially, and the actual merging and pruning stage is postponed to be gradually completed in the subsequent Compilation link.

[0087] In one embodiment, S3. According to the content requirements of marketing recommendation data, select and combine pre-designed second computing logic modules, and use the second computing logic modules to analyze the marketing recommendation source data to generate real-time marketing recommendation data for the specified content, specifically including:

[0088] Obtain the specified content for which real-time marketing recommendation data needs to be generated according to the content requirements of marketing recommendation data. The specified content includes user portraits, product portraits, real-time popularity of products, and user interest in products;

[0089] Determine real-time tasks for data analysis of the marketing recommendation source data according to the specified content. The real-time tasks include: user portrait tasks, product portrait tasks, product popularity list tasks, and user-product collaborative filtering tasks;

[0090] Select the second computing logic module corresponding to each real-time task, and input the marketing recommendation source data into each second computing logic module to obtain real-time marketing recommendation data including user portraits, product portraits, product popularity lists, and user-product relationship pairs where the user's interest in the product is greater than the threshold.

[0091] In this embodiment, as Figure 3 and 4As shown, the marketing recommendation system can customize multiple marketing recommendation rules, which determines the content of the data assets generated by the real-time data warehouse through real-time tasks. For example, the recommendation rules based on collaborative filtering will generate corresponding collaborative filtering tasks, and the relevance between users and products needs to be recorded; for the marketing recommendation that matches tags based on user portraits and product portraits, portrait tasks need to be established to calculate the portraits of products and users respectively; for recommending products based on user access popularity, a popularity list task needs to be established to generate the product popularity ranking, etc. The calculation algorithm for each task is preset as a calculation logic module. By analyzing the services subscribed by users, marketing rules, etc., these calculation logic modules are called to execute corresponding real-time tasks, and the corresponding real-time data assets, that is, real-time marketing recommendation data, can be obtained.

[0092] In one implementation, the method further includes:

[0093] Regarding the real-time marketing recommendation data as real-time data assets, and recording the data lineage relationship between the real-time marketing recommendation data and the marketing recommendation source data to establish a real-time data asset catalog;

[0094] The fourth user manages the real-time data asset catalog, including: managing the same real-time marketing recommendation data requirements and marketing recommendation source data requirements of multiple first users, and promoting hot real-time marketing recommendation data according to the heat map of using real-time marketing recommendation data.

[0095] In this embodiment, as Figure 3 shown by the data assets and as Figure 4 shown by the real-time asset catalog, the establishment process is as Figure 9As shown in the figure, in the asset service management module, it supports functions such as classifying, registering, publishing, and taking off the shelf of real-time data models according to application scenarios, forming a unified real-time data asset catalog. The real-time data asset catalog will include the modeling results of marketing rules and realize the sharing of real-time data assets. Around assets such as real-time tables, tags, and metrics, an integrated management process for registration, publishing / taking off the shelf, and subscription is constructed to make the asset situation visible in all links and the asset list have both in and out, helping business personnel quickly query and analyze data from different channels and dimensions (such as user behavior, product ordering situation, etc.) to quickly obtain a detailed customer portrait. Specifically, it includes the following functions: Asset registration. The asset catalog registers and records the source of the real-time data source, the meaning of fields, the business attributes, classification and grading, relationship mapping, annotations, and sensitive field identification of the calculation result table, so as to help business personnel understand the details of the data set more comprehensively. Through the data lineage of DAG (Directed Acyclic Graph) (including the flow path of data between systems and the processing logic of calculation / mapping), users can intuitively track and trace the conversion of real-time data in the life cycle. In the DAG graph, each task or operation in the data processing process can be used as a node (Node), and the dependency relationship between tasks is used as an edge (Edge). For example, in an ETL (Extract-Transform-Load) process, the process of reading data from the source table, cleaning and transforming it, and finally writing it into the target table can be represented as a DAG; Publishing and taking off the shelf. The asset publisher (the fourth user) completes the pre-publishing of real-time data registration by describing the data usage scenario, configuring the data item explanation, and setting the sensitive data access policy. After the administrator approves it, the data asset opens preview and subscription permissions, avoiding problems such as repeated construction of similar scenarios and inconsistent calculation calibers of the same indicators, improving the indicator consistency, that is, managing the same real-time marketing recommendation data requirements and marketing recommendation source data requirements of multiple first users, and the cold data with low usage rate will be taken off the shelf to improve the data usage efficiency; Subscription and data service. The asset catalog provides an encapsulated API (Application Programming Interface) function to realize data integration. The data user (the first user) can accurately empower the scenarios of data extraction, viewing, and using through the visual interface subscription. In addition, the asset platform (the fourth user) can draw a data heat map according to dimensions such as application scenarios, subscription frequency, and usage effect, and update the catalog organization based on this to promote the promotion of high-quality hot data.

[0096] In one implementation, S4. According to the marketing recommendation data subscription requirements, select and combine the pre-designed third calculation logic module, and use the third calculation logic module to push the real-time marketing recommendation data to the subscribed marketing recommendation terminal, specifically including:

[0097] Obtain the subscribed marketing recommendation terminals, the data push methods subscribed by each marketing recommendation terminal, and the real-time marketing recommendation data content according to the subscription requirements of the marketing recommendation data;

[0098] Select a third computing logic module with the corresponding data push method, and use the third computing logic module to send the corresponding real-time marketing recommendation data content to each marketing recommendation terminal. The third computing logic module implements multiple data push methods based on Flink, and Flink is a unified processing model for stream data and batch data.

[0099] In this embodiment, as Figure 4 shown, the real-time recommendation system focuses on the intelligent operation scenario, based on the real-time data capability, integrates the results of the offline + real-time engine, and forms a standardized intelligent real-time recommendation capability. Each business touchpoint and operation platform uses the real-time intelligent recommendation capability to build a real-time recommendation that matches its own business scenario, as Figure 3 shown, this recommendation process still uses Flink CDC (Change Data Capture) of the data bus to implement.

[0100] In an implementation manner, using the third computing logic module to send the corresponding real-time marketing recommendation data content to each marketing recommendation terminal specifically includes:

[0101] Use the seventh computing logic module to send real-time marketing recommendation data including user portraits, product portraits, product popularity lists, and user-product relationship pairs where the user's interest in the product is greater than the threshold to the marketing recommendation terminal of the first user;

[0102] Use the eighth computing logic module to send real-time marketing recommendation data including the popular products in the product popularity list and the products whose interest of the fifth user is greater than the threshold to the marketing recommendation terminal of the fifth user, and the fifth user is a customer of the first user.

[0103] In this embodiment, in the data recommendation module, by subscribing to the cleaned data capability, based on the recommendation rule combination capability, the target user is output in real time to the application touchpoints (including various marketing tools used by customers or marketers), empowering the front-line marketing scenario. As Figure 4As shown, the marketing recommendation terminal can include the operator (the first user) APP, the public APP, other touchpoints (the customers of the first user, the fifth user), etc. The recommended content varies according to different user identities. Whether to have the permission to send recommendation data can be set in the real-time asset catalog. Different marketing recommendation contents include: real-time recommendation based on collaborative filtering, using Flink to record the categories and products browsed by users, and applying the collaborative filtering algorithm for recommendation based on the user / product relationship; real-time recommendation based on context, calculating the interest degree according to the operations of users on various products, and the calculation rule is that if the operation interval time (such as shopping - browsing < 100s) is determined as an interest event; real-time recommendation based on tags, recording multi-dimensional information of users / products based on user portraits and product portraits, and making recommendations based on the same tags; real-time recommendation based on popularity, through the Flink time window mechanism, counting the real-time popularity at the current time, caching the data in Redis (Remote Dictionary Server), calculating the real-time popularity through the window mechanism, and saving a popularity list once, etc.

[0104] The advantages of this embodiment include: building a highly available data bus capability, realizing high-quality and fast data flow between different systems by being compatible with more than 20 data component interfaces, connecting the data of different systems to achieve free offline or real-time data flow, and meeting the high-speed and stable data synchronization requirements between heterogeneous data sources; building a low-code R & D platform, introducing a storage-computation separation architecture, where users can achieve simple development and management of flow tasks by configuring data source and processing SQL logic node information. The low-code module can automatically construct Flink (a unified processing model for stream data and batch data) flow tasks by identifying the configured data source information nodes and data processing SQL (Structured Query Language) logic. The storage-computation separation architecture optimizes the real-time task status management mechanism, improving resource utilization and job recovery speed; building a real-time data asset catalog management system, establishing an integrated control process for real-time data asset registration, publishing, subscribing, and serving through building a real-time data asset management platform, achieving unified asset catalog and unified management, supporting real-time data tagging by scenario, and supporting accurate data usage in different business scenarios with visualization subscription capabilities; building a flexible rule recommendation system, based on the real-time capabilities of the data middle platform, outputting standardized intelligent recommendation capabilities through multi-dimensional rule combinations, and each business touchpoint and marketing tool matching the recommendation strategy of its own business scenario, meeting the independent selection of recommendation strategies by each touchpoint through multiple modes such as collaborative filtering, context analysis, and portrait.

[0105] Embodiment 2:

[0106] As Figure 2As shown in the figure, the present disclosure provides a real-time marketing recommendation device, which includes:

[0107] A demand unit 1, configured to obtain the marketing recommendation demands customized by the first user in real time. The marketing recommendation demands include marketing recommendation source data demands, marketing recommendation data content demands, and marketing recommendation data subscription demands;

[0108] An acquisition unit 2, connected to the demand unit 1, configured to select and combine a pre-designed first calculation logic module according to the marketing recommendation source data demands, and acquire marketing recommendation source data according to the first calculation logic module;

[0109] A generation unit 3, connected to the acquisition unit 2, configured to select and combine a pre-designed second calculation logic module according to the marketing recommendation data content demands, and analyze the marketing recommendation source data using the second calculation logic module to generate real-time marketing recommendation data of specified content;

[0110] A push unit 4, connected to the generation unit 3, configured to select and combine a pre-designed third calculation logic module according to the marketing recommendation data subscription demands, and push the real-time marketing recommendation data to the subscribed marketing recommendation terminals using the third calculation logic module.

[0111] In an embodiment, the device further includes a back-end development unit, specifically including:

[0112] A UDF development unit, configured to receive user-defined functions (UDFs) developed by the second user according to the marketing recommendation source data demands, marketing recommendation data content demands, and marketing recommendation data subscription demands respectively;

[0113] A calculation logic module development unit, connected to the UDF development unit, configured to generate a first calculation logic module, a second calculation logic module, and a third calculation logic module according to the UDFs respectively.

[0114] In an embodiment, the acquisition unit 2 specifically includes:

[0115] An acquisition rule unit, configured to obtain several real-time data sources for acquiring marketing recommendation source data and offline data sources located in the real-time data warehouse according to the marketing recommendation source data demands, and determine the data acquisition rules for each real-time data source and the data update period for the offline data sources;

[0116] A first selection and combination unit, connected to the acquisition rule unit, configured to receive multiple first calculation logic modules specified by the third user through a visual interface and the connection relationships of each first calculation logic module. The first calculation logic module includes several fourth calculation logic modules for respectively acquiring data from several real-time data sources according to the corresponding data acquisition rules, a fifth calculation logic module for updating data to the offline data source according to the data update period, and a sixth calculation module for acquiring data from the offline data source;

[0117] The Flink SQL task unit, connected to the first selection combination unit, is used to generate a Flink SQL task according to multiple first calculation logic modules and their connection relationships, and collect marketing recommendation source data from a data source according to the Flink SQL task, including collecting stream data from several real-time data sources and batch data from an offline data source. Among them, Flink is a unified processing model for stream data and batch data, and SQL is a structured query language.

[0118] In one embodiment, the Flink SQL task unit specifically includes:

[0119] The container unit is used to dynamically apply for a TaskManager according to the parallelism and resource requirements of the current Flink SQL task, and create a running container for each TaskManager;

[0120] The DFS storage unit, connected to the container unit, is used to store the status data of the Flink job in the remote distributed file system DFS, and multiple TaskManagers share the status data through DFS;

[0121] The fault handling unit, connected to the DFS storage unit, is used to if the Flink job of a certain container fails, the certain container recovers the status data from DFS through status lazy loading and delayed pruning.

[0122] In one embodiment, the generation unit 3 specifically includes:

[0123] The generation rule unit is used to obtain the specified content that needs to generate real-time marketing recommendation data according to the content requirements of the marketing recommendation data. The specified content includes user portraits, product portraits, the real-time popularity of products, and the degree of user interest in products;

[0124] The real-time task unit, connected to the generation rule unit, is used to determine the real-time tasks for data analysis of the marketing recommendation source data according to the specified content. The real-time tasks include: user portrait tasks, product portrait tasks, product popularity list tasks, and user-product collaborative filtering tasks;

[0125] The second selection combination unit, connected to the real-time task unit, is used to select the second calculation logic module corresponding to each real-time task, and input the marketing recommendation source data into each second calculation logic module to obtain real-time marketing recommendation data including user portraits, product portraits, product popularity lists, and user-product relationship pairs with the degree of user interest in products greater than the threshold.

[0126] In one embodiment, the device further includes an asset management unit, which specifically includes:

[0127] An asset catalog unit for taking real-time marketing recommendation data as real-time data assets and recording the data lineage relationship between the real-time marketing recommendation data and the marketing recommendation source data to establish a real-time data asset catalog;

[0128] A management implementation unit connected to the asset catalog unit for a fourth user to manage the real-time data asset catalog, including: managing the same real-time marketing recommendation data requirements and marketing recommendation source data requirements of multiple first users, and promoting hot real-time marketing recommendation data according to the heat map of using real-time marketing recommendation data.

[0129] In one embodiment, the push unit 4 specifically includes:

[0130] A push rule unit for obtaining subscribed marketing recommendation terminals, the data push methods subscribed by each marketing recommendation terminal, and the content of real-time marketing recommendation data according to the marketing recommendation data subscription requirements;

[0131] A third selection and combination unit connected to the push rule unit for selecting a third computing logic module with a corresponding data push method and using the third computing logic module to send the corresponding real-time marketing recommendation data content to each marketing recommendation terminal. The third computing logic module implements multiple data push methods based on Flink, and Flink is a unified processing model for stream data and batch data.

[0132] In one embodiment, the third selection and combination unit specifically includes:

[0133] A seventh combination unit for using a seventh computing logic module to send real-time marketing recommendation data including user portraits, product portraits, product popularity lists, and user-product relationship pairs where the user's interest in the product is greater than a threshold to the marketing recommendation terminal of the first user;

[0134] An eighth combination unit for using an eighth computing logic module to send real-time marketing recommendation data including popular products in the product popularity list and products with a fifth user's interest greater than a threshold to the marketing recommendation terminal of the fifth user, where the fifth user is a customer of the first user.

[0135] Example 3:

[0136] Embodiment 3 of the present disclosure provides a computer-readable storage medium storing a computer program, which when run by a processor, implements the real-time marketing recommendation method as described in Embodiment 1 or implements the real-time marketing recommendation device as described in Embodiment 2.

[0137] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, computer program units, or other data. The computer-readable storage medium includes, but is not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory, or other memory technologies, CD-ROM (Compact Disc Read-Only Memory), digital versatile discs (DVDs), or other optical disc storage, magnetic cassettes, tapes, magnetic disk storage, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0138] In addition, the present disclosure may also provide a computer device including a memory and a processor. A computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes the real-time marketing recommendation method as described in Embodiment 1. The computer device may be the real-time marketing recommendation device as described in Embodiment 2.

[0139] Among them, the memory is connected to the processor. The memory may use flash memory, read-only memory, or other memories, and the processor may use a central processing unit or a single-chip microcomputer.

[0140] Embodiments 1-3 of the present disclosure provide a real-time marketing recommendation method, device, and medium. By analyzing the real-time customized marketing recommendation requirements of a first user, a computing logic module that meets the user's marketing recommendation requirements is selected and combined to implement an integrated marketing maintenance recommendation data service that integrates collecting source data, generating marketing recommendation data, and pushing marketing recommendation content to meet the user's real-time needs, achieving low-code development and flexible marketing recommendation capability combinations, and improving real-time data empowerment for marketing scenarios.

[0141] It can be understood that the above embodiments are merely exemplary embodiments adopted to illustrate the principle of the present disclosure. However, the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and essence of the present disclosure, and these modifications and improvements are also regarded as the protection scope of the present disclosure.

Claims

1. A real-time marketing recommendation method, characterized in that: The method comprises: Obtaining the first user's real-time customized marketing recommendation needs, which include marketing recommendation source data needs, marketing recommendation data content needs, and marketing recommendation data subscription needs; According to the marketing recommendation source data requirements, select and combine the pre-designed first calculation logic module, and collect the marketing recommendation source data according to the first calculation logic module; According to the marketing recommendation data content requirements, select and combine the pre-designed second computing logic module, and use the second computing logic module to analyze the marketing recommendation source data to generate real-time marketing recommendation data of the specified content; According to the marketing recommendation data subscription requirements, a pre-designed third computing logic module is selected and combined, and the third computing logic module is used to push the real-time marketing recommendation data to the subscribed marketing recommendation terminal.

2. The method according to claim 1, characterized in that The method further comprises: Receiving a user-defined function UDF developed by a second user according to marketing recommendation source data requirements, marketing recommendation data content requirements, and marketing recommendation data subscription requirements; A first computing logic module, a second computing logic module and a third computing logic module are generated respectively according to the UDF.

3. The method according to any one of claims 1 to 2, characterized in that: According to the marketing recommendation source data requirements, a pre-designed first calculation logic module is selected and combined, and the marketing recommendation source data is collected according to the first calculation logic module, specifically including: According to the marketing recommendation source data requirements, obtain several real-time data sources and offline data sources located in the real-time data warehouse that need to collect marketing recommendation source data, and determine the data collection rules of each real-time data source and the data update cycle of the offline data source; receiving a plurality of first computing logic modules and connection relationships of the first computing logic modules specified by a third user through a visual interface, wherein the first computing logic modules include a plurality of fourth computing logic modules that respectively collect data from a plurality of real-time data sources according to corresponding data collection rules, a fifth computing logic module that updates data to an offline data source according to a data update cycle, and a sixth computing module that collects data from an offline data source; A Flink SQL task is generated according to multiple first computing logic modules and their connection relationships, and marketing recommendation source data is collected from data sources according to the Flink SQL task, including collecting stream data from several real-time data sources and collecting batch data from offline data sources, wherein Flink is an integrated processing model for stream data and batch data, and SQL is a structured query language.

4. The method according to claim 3, characterized in that According to the Flink SQL task, the marketing recommendation source data is collected from the data source, including: Dynamically apply for a task manager TaskManager based on the parallelism and resource requirements of the current Flink SQL task, and create a running container for each TaskManager; The state data of the Flink job is stored in the remote distributed file system DFS, and multiple TaskManagers share the state data through DFS; If a Flink job of a container fails, the container recovers state data from DFS through state lazy loading and delayed pruning.

5. The method according to any one of claims 1-2, characterized in that: According to the marketing recommendation data content requirements, a pre-designed second computing logic module is selected and combined, and the marketing recommendation source data is analyzed using the second computing logic module to generate real-time marketing recommendation data of specified content, specifically including: According to the marketing recommendation data content requirements, the specified content that needs to generate real-time marketing recommendation data is obtained, including user portraits, product portraits, real-time popularity of products, and user interest in products; Determine the real-time task of data analysis on the marketing recommendation source data according to the specified content. The real-time tasks include: user profiling task, product profiling task, product popularity list task, and user and product collaborative filtering task; Select the second computing logic module corresponding to each real-time task, and input the marketing recommendation source data into each second computing logic module to obtain real-time marketing recommendation data including user portraits, product portraits, product popularity lists, and user-product relationship pairs whose user interest in the product is greater than a threshold.

6. The method according to any one of claims 1-2, characterized in that: The method further comprises: The real-time marketing recommendation data is used as a real-time data asset, and the data lineage relationship between the real-time marketing recommendation data and the marketing recommendation source data is recorded to establish a real-time data asset catalog; The fourth user manages the real-time data asset catalog, including: managing the same real-time marketing recommendation data demands and marketing recommendation source data demands of multiple first users, and promoting hot real-time marketing recommendation data based on a heat map using the real-time marketing recommendation data.

7. The method according to any one of claims 1-2, characterized in that: According to the marketing recommendation data subscription requirements, a pre-designed third computing logic module is selected and combined, and the third computing logic module is used to push the real-time marketing recommendation data to the subscribed marketing recommendation terminal, specifically including: According to the marketing recommendation data subscription requirements, obtain the subscribed marketing recommendation terminals, the data push methods subscribed by each marketing recommendation terminal, and the real-time marketing recommendation data content; Select a third computing logic module with a corresponding data push method, and use the third computing logic module to send the corresponding real-time marketing recommendation data content to each marketing recommendation terminal. The third computing logic module implements multiple data push methods based on Flink. Flink is an integrated processing model for stream data and batch data.

8. The method according to claim 7, characterized in that Using the third computing logic module to send corresponding real-time marketing recommendation data content to each marketing recommendation terminal specifically includes: Use the seventh computing logic module to send real-time marketing recommendation data including user portraits, product portraits, product popularity lists, and user-product relationship pairs whose user interest in products is greater than a threshold to the marketing recommendation terminal of the first user; The eighth computing logic module is used to send real-time marketing recommendation data including popular products in the product popularity list and products whose interest level of the fifth user is greater than a threshold to the marketing recommendation terminal of the fifth user, where the fifth user is a customer of the first user.

9. A real-time marketing recommendation device, characterized in that: The device comprises: A demand unit, used to obtain a marketing recommendation demand customized in real time by the first user, the marketing recommendation demand including a marketing recommendation source data demand, a marketing recommendation data content demand and a marketing recommendation data subscription demand; A collection unit connected to the demand unit, configured to select and combine a pre-designed first calculation logic module according to the marketing recommendation source data demand, and collect the marketing recommendation source data according to the first calculation logic module; A generating unit connected to the collecting unit, configured to select and combine a pre-designed second computing logic module according to the content requirements of the marketing recommendation data, and use the second computing logic module to analyze the marketing recommendation source data to generate real-time marketing recommendation data of specified content; The push unit is connected to the generation unit and is used to select and combine the pre-designed third computing logic module according to the marketing recommendation data subscription requirements, and use the third computing logic module to push the real-time marketing recommendation data to the subscribed marketing recommendation terminal.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the real-time marketing recommendation method according to any one of claims 1 to 8 is implemented.

Images