A method and computer system for BOP shot placement

Abstract

This invention relates to the field of data processing technology, and more particularly to a BOP (Browser-Operated Programming) click sending method. This method is used in a BOP system and employs a network load balancer as the data interface. Upon receiving a user click request, the method first sends the user click request to a preset service cluster. Then, in response to the triggering of a preset channel click script, it writes the user click requests from the corresponding service cluster into a distributed stream processing platform for data grouping. Finally, it sends the grouped user click requests to their respective databases for storage. Compared to existing technologies, this invention overcomes the problem of difficulty in integrating and storing click data with a BOP system.

Description

Technical Field

[0001] This invention relates to the field of data processing technology. More specifically, this invention relates to a BOP (Browser Layout Programming) click sending method and computer system. Background Technology

[0002] BOP (Business Operation Platform) refers to a comprehensive business management system specifically designed for companies in the advertising industry. It aims to support the company's daily operations and typically includes modules such as order management, inventory management, customer relationship management, and financial management. Click-through rate (CTR) is the ratio of the number of times a piece of content on a website page is clicked to the number of times it is displayed; it is calculated as clicks / displays × 100%. It is a percentage used to reflect the level of attention received by a particular piece of content on a webpage.

[0003] Chinese patent application CN104091276B discloses a method, related apparatus, and system for online analysis of clickstream data. The method described in this patent application can obtain click-through rates from web pages, perform data analysis, and store the data. However, its click-through rate sending method is only applicable to advertising servers (Ad Servers). The database of an advertising server typically consists of MySQL, MongoDB, Redis, or another single database. Due to limitations in the algorithm of the click-through rate sending method and the data storage architecture of the advertising server, directly applying this method to a BOP (Browser Open Platform) system would lead to data storage chaos and data transmission overload, damaging the original functions of the BOP system (order management, inventory management, customer relationship management, and financial management, etc.).

[0004] Therefore, existing click sending methods are difficult to integrate and store with BOP systems. Summary of the Invention

[0005] To address the technical problem that existing click sending methods are difficult to integrate and store with BOP systems, this invention discloses a BOP click sending method and computer system.

[0006] In a first aspect, this invention discloses a BOP (Browser Open Platform) click delivery method for use in a BOP system, employing a network load balancer as the data interface. The method includes:

[0007] In response to receiving a user click request, the user click request is sent to the preset service cluster;

[0008] In response to the triggering of the preset channel click script, the user click requests in the corresponding service cluster are written to the distributed stream processing platform for data grouping;

[0009] The grouped user click requests are sent to the corresponding databases for storage.

[0010] Beneficial Effects: This invention employs a network load balancer (ELB) as the data interface, enabling the distribution / allocation of user click requests to multiple pre-defined service clusters. This ensures even information distribution and avoids single-point overload. Furthermore, user click requests from the corresponding service clusters are written into a distributed stream processing platform for data grouping, allowing for parallel data processing and improved throughput. Regarding data storage, this invention stores data according to the data grouping of user click requests, ensuring reasonable storage based on data characteristics. Compared to existing technologies, this invention overcomes the technical challenge of integrating existing click-feeding technologies with BOP systems by setting up a network load balancer and a distributed stream processing platform. Moreover, through grouped storage, it overcomes the problem of reasonable storage in BOP systems by existing click-feeding technologies.

[0011] Preferably, the service cluster includes at least an Nginx-BigData service cluster, a click service cluster, and an asynchronous click service cluster.

[0012] Beneficial effects: The Nginx-BigData service cluster is used to identify the type and source of user click requests, the click service cluster is used to handle core click logic (such as verification, billing and real-time response), and the asynchronous click service cluster is used to handle non-real-time tasks (such as log recording and data analysis result feedback).

[0013] Preferably, the Nginx-BigData service cluster processes user click requests and then forwards them to the backend click service cluster.

[0014] Preferably, after sending the user click request to a preset service cluster, the method of the present invention further includes:

[0015] Call the click service cluster to extract the click parameters of the user's click request;

[0016] Validate the click parameters;

[0017] If the validation passes, the click parameters will be used as the data to be written.

[0018] Preferably, the distributed stream processing platform uses Kafka, and the database includes at least MongoDB, a Sourcetag-based database, Cassandra, OBS (Object Storage Service), Aurora (Amazon Aurora), and a project management repository.

[0019] Beneficial effects: Kafka boasts extremely high data throughput, capable of processing millions of messages per second, making it well-suited for delivering click data on web pages. The database consists of MongoDB, a Sourcetag-based database, Cassandra, OBS, Aurora, and a project management repository, which can specifically store data related to clicks.

[0020] Preferably, before sending the grouped user click requests to their respective databases for storage, the method of the present invention further includes:

[0021] Write user click requests that are not grouped by Kafka to OBS.

[0022] Beneficial effect: Writing user click requests that are not grouped by Kafka to OBS enables automatic data backup and archiving.

[0023] Preferably, the grouped user click requests are sent to the corresponding databases for storage, including:

[0024] In response to a pre-configured Flume (log collection system) trigger, user click requests grouped by Kafka are persistently stored in Cassandra.

[0025] Preferably, in addition to sending the grouped user click requests to the corresponding databases for storage, the method of the present invention further includes:

[0026] User click requests grouped by Kafka are sent to a real-time data analysis platform for real-time data analysis to obtain real-time data.

[0027] Real-time data is sent to MongoDB for storage.

[0028] Preferably, after receiving the user's click request, the method of the present invention further includes:

[0029] Retrieve offers that match the user's click request;

[0030] Monitor offer access volume, processing time, or success rate in real time and store the data in MongoDB.

[0031] In a second aspect, the present invention discloses a computer system including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the BOP sending click method described in the first aspect is implemented.

[0032] The beneficial effects of this invention are as follows:

[0033] (1) Compared with the prior art, the method of the present invention overcomes the technical difficulty of the existing click sending technology in connecting with the BOP system by setting up a network load balancer and a distributed stream processing platform, and overcomes the problem of the existing click sending technology in achieving reasonable storage in the BOP system by group storage.

[0034] (2) Compared with the prior art, the Kafka used in this invention can handle extremely high data throughput, processing millions of messages per second, and can meet the needs of transmitting click data in web pages. The database of this invention consists of MongoDB, a Sourcetag-based database, Cassandra, OBS, Aurora, and a project management library. These databases can store data related to click data in a targeted manner.

[0035] (3) Compared with the prior art, the present invention writes user click requests that are not grouped by Kafka into OBS, which can realize automatic data backup and archiving. Attached Figure Description

[0036] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0037] Figure 1 This is a flowchart of the BOP click sending method in Embodiment 1 of the present invention;

[0038] Figure 2 This is a data flow transmission diagram in Embodiment 1 of the present invention;

[0039] Figure 3 This is a schematic diagram of the computer system in Embodiment 2 of the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0042] Example 1

[0043] like Figure 1As shown, this embodiment discloses a BOP (Browser Open Platform) click delivery method, including:

[0044] S10: In response to receiving a user click request, send the user click request to the preset service cluster.

[0045] In this embodiment, after a user clicks on a webpage or other client-side channel, data is distributed through a network load balancer (ELB) as the data interface, and then step S10 is executed. Multiple service clusters are preset.

[0046] S20: In response to the triggering of the preset channel click script, the user click request in the corresponding service cluster is written to the distributed stream processing platform for data grouping.

[0047] In this embodiment, the aforementioned distributed stream processing platform uses Kafka, a distributed stream processing platform with extremely high data throughput, capable of processing millions of messages per second and adapting to the transmission of click data in web pages or clients.

[0048] S30: Send the grouped user click requests to the corresponding databases for storage.

[0049] In this embodiment, multiple databases are set up. After the user clicks on the request and the data is grouped, the data types of the groups are different, and each database is used to store different types of data.

[0050] Through steps S10-S30 above, this embodiment uses a network load balancer as the data interface to distribute / allocate user click requests to multiple preset service clusters, ensuring even distribution of information and avoiding single-point overload. Utilizing Kafka's characteristics, high-concurrency data processing is achieved, improving data throughput and making it possible to interface click requests with the BOP system (i.e., without affecting other built-in BOP functions). Regarding data storage, this embodiment stores data according to the data grouping of user click requests, enabling reasonable storage based on data characteristics.

[0051] Compared with existing technologies, the method of this embodiment overcomes the problem that existing click sending technologies are difficult to store reasonably in a BOP system.

[0052] Combination Figure 2 As shown, the above service clusters include at least the Nginx-BigData service cluster, the click service cluster, and the asynchronous click service cluster.

[0053] Preferably, the Nginx-BigData service cluster processes user click requests and then forwards them to the backend click service cluster.

[0054] Specifically, Nginx typically acts as a web server to receive client requests, while BigData refers to large amounts of data. In this embodiment, the Nginx-BigData service cluster is used to identify the type and source of user click requests (clicked URLs). Multiple click service clusters can be set up, mainly used to handle core click logic, such as verifying whether a click is valid, and counting clicks as click responses. Asynchronous click service clusters are mainly used to handle non-real-time tasks, such as extracting user click requests from log records or extracting click feedback from data analysis results.

[0055] In this embodiment, the output data of both the Nginx-BigData service cluster and the asynchronous click service cluster are sent to the click service cluster for data processing of the core click logic.

[0056] Furthermore, after sending the user click request to the backend click service cluster, the method in this embodiment also includes:

[0057] Call the click service cluster to extract the click parameters of the user's click request;

[0058] Validate the click parameters;

[0059] If the validation passes, the click parameters will be used as the data to be written.

[0060] In this embodiment, the click parameters may include click frequency, source, and click type. The verification process is based on preset rules.

[0061] For example, by setting a click frequency threshold, the system can verify whether "brushing" or "traffic fraud" occurs. If the click frequency in the click parameters exceeds the threshold, it indicates that the user is engaging in "brushing" or "traffic fraud" on the webpage or client, and the verification fails. Otherwise, the verification passes. By setting a whitelist of clickable URLs or a whitelist of client applications, the range of click sources can be limited. If the source data in the click parameters exceeds the whitelist range, it indicates that the user's click request is from an illegal source.

[0062] The above technical solution can filter out non-compliant or illegal user click requests, ensuring the authenticity of the click data.

[0063] Furthermore, the databases involved in the method of this embodiment include MongoDB, Sourcetag-based databases, Cassandra, OBS, Aurora, and project management libraries.

[0064] MongoDB is primarily used to store real-time statistics on ad clicks and conversions. Its high-performance query and write capabilities ensure accurate and timely ad click and conversion statistics, guaranteeing control over Cap. A Sourcetag-based database is a custom data warehouse responsible for retrieving data from Kafka consumer groups, cleaning and transforming the data, and then passing the processed data to the next stage. Sourcetags represent specific processing logic such as adding source tags to data, used to identify the data's origin or perform data classification. Cassandra is mainly used to store ad click data within a certain time period, which can be queried and accessed for ad conversions, ensuring that ad conversions are correctly attributed to specific ad clicks. OBS archives all raw ad click and conversion data for easy retrieval. Aurora refers to Amazon Aurora, a commercial-grade relational database service from Amazon compatible with MySQL and PostgreSQL. In this embodiment, it is mainly used to store business data related to transaction relationships; when an ad click occurs, transaction relationship matching is required. Project Manager interfaces with MongoDB to provide data services to the product management system.

[0065] Compared to existing technologies, the database in this embodiment consists of MongoDB, a Sourcetag-based database, Cassandra, OBS, Aurora, and a project management library, which can store data related to click data in a targeted manner, thereby improving the efficiency of data output and storage.

[0066] Furthermore, before sending the grouped user click requests to the corresponding databases for storage in step S30, the method in this embodiment also includes:

[0067] Write user click requests that are not grouped by Kafka to OBS.

[0068] The above solution enables automatic backup and archiving of data related to click requests, preventing data loss due to power outages or other system failures.

[0069] Furthermore, the grouped user click requests are sent to the corresponding databases for storage, including:

[0070] In response to a default Flume trigger, user click requests grouped by Kafka are persistently stored in Cassandra.

[0071] It should be noted that Flume, as mentioned above, refers to a distributed log collection system that can efficiently collect, aggregate, and move large amounts of log data. Figure 2 In this context, RTDAP stands for Real-Time Data Analytics Platform, which is a Spark task used for near real-time statistical calculation of click and conversion volumes and writing them to MongoDB. This data is then used by the click service and conversion receiving service to implement cap control capabilities. ETL stands for Standards-Based Process, a data cleaning and transformation tool used to extract and clean ad click and conversion data to create more complete and standardized data. This data is then used in subsequent business processes such as billing and reconciliation.

[0072] In this embodiment, Flume is mainly used to collect click persistent databases and store them in Cassandra, thereby achieving efficient distributed storage of massive click data.

[0073] To respond to user clicks in real time and store real-time data, this embodiment of the method further includes sending the grouped user click requests to the corresponding databases for storage:

[0074] User click requests grouped by Kafka are sent to a real-time data analysis platform for real-time data analysis to obtain real-time data.

[0075] Real-time data is sent to MongoDB for storage.

[0076] Unlike the data storage process in steps S10-S40 described above, the method in this embodiment, after receiving the user's click request, further includes the following parallel steps:

[0077] Retrieve offers that match the user's click request.

[0078] Monitor offer access volume, processing time, or success rate in real time and store the data in MongoDB.

[0079] In this embodiment, the offer includes, but is not limited to, information such as discounts, promotions, and special-offer products. When a user clicks a specific button on the webpage / client, they are redirected to the corresponding offer. During this process, this real-time method acquires click-related data such as the number of times the offer is accessed, processing time, or success rate, and stores this data in MongoDB.

[0080] The above technical solution enables real-time monitoring of offer information related to user click requests, assisting in the analysis and processing of click data.

[0081] Based on the above technical description, the method of this embodiment has at least the following technical advantages:

[0082] (1) By setting up a network load balancer and a distributed stream processing platform, the webpage click data and the BOP system's sent click data were connected.

[0083] (2) A data processing and storage architecture with high data processing efficiency was designed. By using group storage, the problem that existing click technology is difficult to achieve reasonable storage in BOP system was overcome.

[0084] (3) It has extremely high data throughput and can process millions of messages per second, which can meet the current needs of web page click data transmission.

[0085] (4) The storage architecture designed by the method in this embodiment has multiple storage insurances, and the data sent using the method in this embodiment is not easily lost.

[0086] Example 2

[0087] like Figure 3 As shown, this embodiment discloses a computer system, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the BOP sending click method described in Embodiment 1 is implemented.

[0088] The system also includes other components well known to those skilled in the art, such as communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0089] In this embodiment, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions that can be stored or otherwise maintained by such a computer-readable medium.

[0090] In the description of this specification, "multiple" means at least two, such as two, three or more, etc., unless otherwise expressly and specifically defined.

[0091] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. A BOP (Browser Open Platform) click delivery method, characterized in that, For use in a BOP system and with a network load balancer as the data interface, the method includes: In response to receiving a user click request, the user click request is sent to a preset service cluster; In response to the triggering of the preset channel click script, the user click requests in the corresponding service cluster are written to the distributed stream processing platform for data grouping; The grouped user click requests are sent to the corresponding databases for storage. The distributed stream processing platform uses Kafka, and the database includes at least MongoDB, a Sourcetag-based data warehouse, Cassandra, OBS, Aurora, and a project management library. Before sending the grouped user click requests to their respective databases for storage, the method further includes: writing user click requests that have not been grouped by Kafka into the OBS; The grouped user click requests are then sent to their respective databases for storage, including: In response to a pre-defined Flume trigger, user click requests that have passed through the Kafka group are persistently stored in Cassandra; The method further includes sending the grouped user click requests to the corresponding databases for storage, and sending the user click requests grouped by Kafka to a real-time data analysis platform for real-time data analysis to obtain real-time data; and sending the real-time data to MongoDB for storage. After receiving a user click request, the method further includes: Query offers that match the user's click request; monitor the access volume, processing time, or success rate of the offer in real time and store them in MongoDB; Among them, the Sourcetag-based data warehouse is a custom data warehouse responsible for obtaining data from the Kafka consumer group, cleaning and transforming the data, and then passing the processed data to the next stage; Sourcetag represents the data source tagging processing logic, used to identify the source of the data or to perform data classification operations.

2. The BOP click delivery method according to claim 1, characterized in that, The service cluster includes at least an Nginx-BigData service cluster, a click service cluster, and an asynchronous click service cluster.

3. The BOP click delivery method according to claim 2, characterized in that, The Nginx-BigData service cluster processes the user click request and forwards it to the backend click service cluster.

4. The BOP click delivery method according to claim 2, characterized in that, After sending the user click request to a preset service cluster, the method further includes: Invoke the click service cluster to extract the click parameters of the user's click request; The click parameters are validated; If the verification passes, the click parameters will be used as data to be written.

5. A computer system, characterized in that, It includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the BOP sending click method according to any one of claims 1-4 is implemented.

Images