Data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a storage medium, and relates to the technical field of data processing. Comprising the following steps: determining the value of each candidate target product according to data of each candidate target product recorded in a database; determining an original recall queue according to the value of each candidate target product; the original recall queue comprises a plurality of candidate target products, and the candidate target products are sorted according to values of the candidate target products; according to the real-time operation state data of the bidding platform and a queue length regulation and control strategy, the regulation and control length of an original recall queue is determined; and according to the regulation and control length and a preset truncation strategy, truncation processing is performed on the original recall queue to obtain a target recall queue. According to the method, the queue length of the original recall queue is triggered and dynamically adjusted according to the real-time operation state data of the bidding platform, different flow scene requirements can be met, the resource utilization rate is improved, the performance bottleneck of the bidding platform is avoided, and stable operation and quick response of the platform are guaranteed.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] In real-time bidding platforms, multiple users typically participate in the bidding simultaneously. Products are submitted and enter the bidding process. The bidding system generates a product recall queue based on factors such as the quantity and value of the products offered. The products in the recall queue are then finely screened to select the final products for delivery. Generally, an excessively long product recall queue can negatively impact system performance. Therefore, appropriately controlling the queue length can improve system processing performance to some extent.

[0003] In existing technologies, the length of the recall queue is usually not limited, which leads to excessive consumption of system resources and seriously affects system processing performance. Summary of the Invention

[0004] The purpose of this application is to address the shortcomings of the prior art by providing a data processing method, apparatus, electronic device, and storage medium to improve the resource utilization of the bidding platform and ensure its stable operation and rapid response.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows: In a first aspect, embodiments of this application provide a data processing method, including: The value of each candidate target product is determined based on the data of each candidate target product recorded in the database; The original recall queue is determined based on the value of each candidate target product; the original recall queue includes multiple candidate target products, and each candidate target product is sorted according to its value. The adjustment length of the original recall queue is determined based on the real-time operating status data of the bidding platform and the queue length adjustment strategy. Based on the adjusted length and the preset truncation strategy, the original recall queue is truncated to obtain the target recall queue.

[0006] Secondly, embodiments of this application also provide a data processing apparatus, including: a determining module and a processing module; The determining module is used to determine the value of each candidate target product based on the data of each candidate target product recorded in the database; The determining module is used to determine the original recall queue based on the value of each candidate target product; the original recall queue includes multiple candidate target products, and each candidate target product is sorted according to its value. The determining module is used to determine the control length of the original recall queue based on the real-time operating status data of the bidding platform and the queue length control strategy. The processing module is used to truncate the original recall queue according to the controlled length and the preset truncation strategy to obtain the target recall queue.

[0007] Thirdly, embodiments of this application provide an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the data processing method provided in the first aspect.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the data processing method as provided in the first aspect.

[0009] The beneficial effects of this application are: This application provides a data processing method, apparatus, electronic device, and storage medium, comprising: determining the value of each candidate target product based on data of each candidate target product recorded in a database; determining an original recall queue based on the value of each candidate target product; the original recall queue including multiple candidate target products, each candidate target product being sorted according to its value; determining the adjustment length of the original recall queue based on real-time operating status data of the bidding platform and a queue length adjustment strategy; and truncating the original recall queue according to the adjustment length and a preset truncation strategy to obtain a target recall queue. This method dynamically adjusts the queue length of the original recall queue based on real-time operating status data of the bidding platform, which can adapt to different traffic scenarios, improve resource utilization, avoid performance bottlenecks of the bidding platform, and ensure stable operation and rapid response of the platform.

[0010] Secondly, when calculating product value, smoothing the calculation by integrating multi-dimensional data can improve the accuracy of product value calculation and the precision of subsequent truncation processing.

[0011] In addition, when performing truncation, products are grouped by advertisers, and then truncation is carried out by combining value ranking, prioritizing the retention of high-value products, and randomly shuffling low-value products. This ensures fair exposure of advertisers' products while protecting the overall revenue and commercial value of the platform, minimizing the loss of high-value products, and promoting product diversity. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This application provides a schematic diagram of the architecture of a data processing system. Figure 2 Flowchart of the data processing method provided in the embodiments of this application Figure 1 ; Figure 3 Flowchart of the data processing method provided in the embodiments of this application Figure 2 ; Figure 4 A schematic diagram of the overall framework of a value smoothing update method provided in this application embodiment; Figure 5 A flowchart illustrating a smoothing estimation algorithm provided in an embodiment of this application; Figure 6 Flowchart of the data processing method provided in the embodiments of this application Figure 3 ; Figure 7 Flowchart of the data processing method provided in the embodiments of this application Figure 4 ; Figure 8 This application provides a schematic diagram of a dynamic queue length control process. Figure 9 Flowchart of the data processing method provided in the embodiments of this application Figure 5 ; Figure 10 Flowchart of the data processing method provided in the embodiments of this application Figure 6 ; Figure 11 This is a schematic diagram of a truncation processing flow provided in an embodiment of this application; Figure 12 A schematic diagram of a data processing apparatus provided in an embodiment of this application; Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0014] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0015] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0016] First, let's explain some key terms that may be involved in this plan: DSP (Demand-Side Platform): A demand-side platform, an online advertising bidding platform that purchases ad space through real-time bidding and combines user data to achieve precise targeting and improve campaign efficiency.

[0017] eCPM (effective Cost Per Mille): Effective revenue per thousand impressions, a core metric for measuring the value of an ad and guiding resource allocation.

[0018] Ad Recall Queue: In real-time bidding, the DSP recalls a set of all eligible ad orders from the index storage engine based on targeting criteria, which are then used as bidding candidates.

[0019] Queue truncation control: A method for dynamically controlling the length of the original ad retrieval queue. The DSP adjusts the recall quantity and ratio reasonably based on the system status and the advertiser's delivery status.

[0020] In real-time bidding advertising platforms, multiple advertisers participate in the bidding simultaneously, with their ad placements entering the bidding process. An advertiser's strategy, quantity, and value significantly impact the competitive outcome. Generally, advertisers with a larger quantity and higher quality ad placements are more likely to succeed. However, due to limitations in strategy and brand strength, not all advertisers can maintain a high standard. DSP systems have a limited number of ad placements that can be recalled at a time, and traditional bidding retrieval systems control the length of the recall queue to avoid negative impacts on the system from excessively long queues.

[0021] Traditional truncation methods may over-truncate advertisers with insufficient ad slots, making it difficult for them to gain exposure and a chance to win, resulting in "insufficient and unfair bidding." Meanwhile, during peak traffic periods, excessively long recall queues can lead to system performance bottlenecks, timeouts, or even crashes. While existing hard truncation alleviates performance issues, it exacerbates the squeeze on smaller advertisers by top advertisers and may mistakenly delete high-value ad orders, causing revenue losses for the platform.

[0022] Current technologies lack the ability to dynamically adjust based on real-time system status, making it impossible to flexibly adjust the ad recall queue. Therefore, the DSP retrieval and recall phase needs to create a fairer bidding environment, ensuring a reasonable distribution of advertisers and ad orders, mitigating unfairness, and balancing system performance with maximizing ad value.

[0023] Existing technology 1: Existing bidding advertising systems typically employ unlimited recall queue management. During the ad retrieval phase, the system recalls all eligible ad candidates based on user requests and matching rules, without limiting the queue length. The core idea is "the more candidates recalled, the better," believing that more candidate ads will provide better bidding results and user experience.

[0024] Disadvantages of existing technology 1: 1. System performance issues: During peak traffic periods, the number of ad orders increases or the number of low-value ad orders increases, resulting in an excessively long recall queue, which seriously affects system processing performance, causing response delays, or even timeouts or crashes.

[0025] 2. Excessive resource consumption: In the bidding chain, an excessively long queue affects the real-time performance of algorithm sorting and inference calculations, consumes a large amount of computing and memory resources, and can easily cause system resource bottlenecks under high concurrency.

[0026] 3. Lack of flexible adjustment capability: It cannot dynamically adjust according to the real-time status of the system, and uses the same strategy under different traffic scenarios, which lacks flexibility.

[0027] To address the performance issues of Existing Technology 1, Existing Technology 2 is proposed: The system employs a hard truncation control to manage the recall queue length. This scheme sets a fixed queue length threshold; when the number of recalled ads exceeds this threshold, excess ads are truncated, and only the first N ads are retained for further processing.

[0028] Disadvantages of existing technology 2: 1. Top advertiser squeeze issue: Top advertisers have a large number of ad orders and easily occupy most of the queue. When the search is truncated according to the search order, a large number of ad orders from smaller advertisers are truncated, resulting in unfair competition and customer complaints.

[0029] 2. Loss of high-value ad orders: Hard truncation may mistakenly delete high-value ad orders, resulting in loss of exposure, consumption indicators and revenue, and affecting the platform's commercial value.

[0030] 3. Lack of dynamic adjustment capability: Fixed cutoff thresholds cannot adapt to changes in traffic flow at different times. They may over-cut off during off-peak hours and under-cut off during peak hours.

[0031] 4. Damage to the advertiser ecosystem: Long-term unfair cutoffs may lead to the loss of small advertisers, damaging the diversity and healthy development of the advertising ecosystem.

[0032] Based on this, this solution aims to address the following technical issues: 1. Resolve system performance degradation caused by excessively long recall queues: During peak traffic periods, the recall queue of the bidding advertising system expands abnormally, leading to a decline in system performance and business metrics, and affecting user experience and stability.

[0033] 2. Address the issues of advertiser squeeze and loss of high-value ad orders caused by hard truncation: The existing hard truncation scheme may cause top advertisers to squeeze out smaller advertisers, preventing their ad orders from getting sufficient exposure; at the same time, it may mistakenly truncate high-value ad orders, causing damage to key metrics and affecting platform revenue.

[0034] 3. Achieve dynamic and elastic control of the recall queue: Existing solutions lack the ability to dynamically adjust the length of the recall queue based on the real-time operating status of the system (such as timeout rate and failure rate), which cannot adapt to the needs of different traffic scenarios, resulting in low resource utilization or increased system risk.

[0035] It is worth noting that this solution can play a crucial role in various scenarios requiring resource allocation and matching. These scenarios all employ a similar "recall-ranking-bidding-allocation" technical process, achieving efficient resource distribution by quickly filtering high-value products from massive candidate data.

[0036] This solution is not limited to handling ad bidding recall queues; it can also be applied to e-commerce product placement, news feed and media resource recommendations, and education and paid knowledge platforms. For example, it can recall articles and videos that users might be interested in from a massive content pool, and then rank them based on creator bids and content quality. Another example is how educational platforms can use bidding to determine the display priority of courses on the homepage and category pages, increasing the exposure of high-quality courses.

[0037] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0038] Figure 1 This is a schematic diagram of the architecture of a data processing system provided in an embodiment of this application, such as... Figure 1 As shown, the system mainly consists of three core modules: a value update module, a system monitoring and control module, and a multi-strategy dynamic control module; the link is divided into offline link and online link.

[0039] The online process, or real-time bidding process, begins with a user request. The retrieval module generates an initial recall queue. Subsequently, multiple systems determine whether to truncate the queue based on multi-dimensional strategies. If truncation is required, a value- and fairness-based truncation strategy (implemented by a multi-strategy dynamic adjustment module) is executed, ultimately generating an optimized target recall queue for subsequent bidding stages.

[0040] The offline link supports dynamic decision-making in the online link. It calculates product value based on exposure and consumption data (implemented by the value update module), and dynamically adjusts the truncation length of the original recall queue in conjunction with monitoring the operational status of the bidding platform (implemented by the system monitoring and control module).

[0041] Among them, the value update module: relying on the retrieval platform, feature storage engine and smoothing calculation task, calculates the value of all invested products, and uses the quantified value for subsequent truncation strategies.

[0042] The system monitoring and control module periodically reads real-time reports from the bidding platform, including timeout rate, anomaly rate, bidding time, and current queue size. By analyzing the anomaly and timeout rates, it dynamically adjusts the maximum length of the allowed recall queue. When the system load is high or there are many anomalies, the number of recalls is reduced in increments; after the system recovers, queue length control is automatically restored, achieving intelligent self-adaptation.

[0043] Multi-strategy dynamic adjustment module: Responsible for the dynamic truncation management of the original recall queue in real-time bidding. Based on the recall management strategy based on value fairness and the adjustment strategy based on system state, it decides whether to adjust the queue and how to adjust it, so as to achieve system state balance with minimal loss of revenue.

[0044] Figure 2 Flowchart of the data processing method provided in the embodiments of this application Figure 1 The subject executing this method can be a computer device, and the computer device may be equipped with the aforementioned... Figure 1 The system shown, such as Figure 2 As shown, the method includes: S101. Determine the value of each candidate target product based on the data of each candidate target product recorded in the database.

[0045] This solution will use the scenario of processing the bidding recall queue for ad orders as an example to illustrate the steps involved in implementing this solution. Of course, as explained above, in practical applications, the target product can be an e-commerce recommended product, an educational product, etc.

[0046] The database can record data on all candidate target products for all advertisers participating in the bidding, such as data on all candidate ad orders. The data recorded in the database is updated in real time. As advertisers add new requests or products are removed and added, the candidate target products recorded in the database also change in real time.

[0047] The value of each candidate target product can be determined in real time based on its consumption and exposure data. Since the consumption and exposure data of candidate target products are cumulatively updated in real time, the value of each candidate target product is also updated in real time.

[0048] S102. Determine the original recall queue based on the value of each candidate target product; the original recall queue includes multiple candidate target products, and each candidate target product is sorted according to its value.

[0049] At any given moment, the initial recall queue can be determined based on the value of each candidate target product. The length of the initial recall queue is typically unlimited or pre-defined. The initial recall queue can contain all candidate target products recorded in the database, or it can contain multiple candidate target products initially selected from all candidate target products.

[0050] The candidate target products in the original recall queue can be sorted according to value, for example, the higher the value, the higher the ranking.

[0051] S103. Determine the control length of the original recall queue based on the real-time operating status data of the bidding platform and the queue length control strategy.

[0052] When it's an ad bidding scenario, the bidding platform can be the aforementioned DSP, i.e., an ad bidding platform. Real-time operational status data of the bidding platform can reflect its performance. Using this data to guide adjustments to the original recall queue length can effectively prevent performance bottlenecks or timeout crashes.

[0053] In some embodiments, the queue length control strategy is used to control the specific determination of the control length of the original recall queue. That is, by setting a reasonable control strategy, the control length of the determined original recall queue can be made more in line with the actual operation of the bidding platform, thereby achieving dynamic control.

[0054] S104. Based on the control length and the preset truncation strategy, the original recall queue is truncated to obtain the target recall queue.

[0055] Optionally, the control length can determine how much data needs to be truncated from the original recall queue, while a preset truncation strategy can specify how to truncate and which data should be truncated, achieving precise truncation while ensuring fairness and maximizing benefits. For example, assuming the original recall queue length is 1000 and the control length is 100, it can be determined that 100 candidate target products need to be truncated from the original recall queue, that is, only 900 candidate target products should be retained. The preset truncation strategy can be used to determine which 900 candidate target products to retain and which 100 to truncate and delete.

[0056] By truncating the queue, the length of the original recall queue can be limited, thereby preventing an excessively long queue from impacting the performance of the bidding platform.

[0057] In summary, the data processing method provided in this embodiment includes: determining the value of each candidate target product based on the data of each candidate target product recorded in the database; determining an original recall queue based on the value of each candidate target product; the original recall queue includes multiple candidate target products, which are sorted according to their value; determining the adjustment length of the original recall queue based on the real-time operating status data of the bidding platform and the queue length adjustment strategy; and truncating the original recall queue according to the adjustment length and a preset truncation strategy to obtain the target recall queue. This method dynamically adjusts the queue length of the original recall queue based on the real-time operating status data of the bidding platform, which can adapt to different traffic scenarios, improve resource utilization, avoid performance bottlenecks of the bidding platform, and ensure the stable operation and rapid response of the platform.

[0058] Figure 3 Flowchart of the data processing method provided in the embodiments of this application Figure 2 Optionally, in step S101, the value of each candidate target product is determined based on the data of each candidate target product recorded in the database, including: S201. Based on the identifier of each candidate target product, determine the advertiser and product placement position of each candidate target product.

[0059] Typically, a database can store the data for each candidate target product in the form of a list, for example, one candidate target product per row, along with information about the advertiser and the corresponding ad placement. In other words, it records which advertiser applied for the ad placement and in which ad placement it should be placed. Taking an ad flyer as an example, the advertiser and ad placement can be determined based on the ad flyer's identifier.

[0060] S202. Obtain multi-dimensional exposure data and consumption data for each candidate target product from the database.

[0061] Meanwhile, the database can also record product exposure data and consumption data for each candidate target product under different dimensions.

[0062] The dimensions include one or more of the following: product dimension, advertiser dimension, and product placement dimension. Exposure data can refer to the number of times a product is advertised or displayed. Cost data can refer to the number of clicks or redirects after clicks, or the amount of money spent after a click-through, etc.

[0063] For example, product-level exposure can refer to the product's own cumulative exposure, while advertiser-level exposure can refer to the cumulative exposure of the advertiser to which the product belongs, that is, the cumulative exposure of all target products advertised by that advertiser. For example, the cumulative exposure of all ad placements placed by an advertiser. Product placement-level exposure can refer to the cumulative exposure of the placement, that is, the cumulative exposure of all target products advertised in that placement. For example, the cumulative exposure of all ad placements in that placement.

[0064] S203. Determine the value of each candidate target product based on its multi-dimensional exposure and consumption data.

[0065] In some embodiments, the value of all candidate target products can be recalculated at preset intervals, and the cumulative value field in the corresponding index data of each candidate target product in the database can be updated incrementally in real time. Taking an advertising order as an example, calculating the value of an advertising order can refer to calculating the cumulative eCPM value of the advertising order. Of course, different products have different value measurement metrics, and are not limited to using the cumulative eCPM value.

[0066] In this embodiment, the exposure data and consumption data of each candidate target product in the above-mentioned multiple dimensions can be combined. Through smooth estimation, based on three levels of granularity (product dimension, advertiser dimension, and advertiser location dimension), and through hierarchical compensation and weighted fusion, the product value in sparse exposure scenarios can be calculated, thereby improving the accuracy of the value calculation results.

[0067] Optionally, in step S202, the value of each candidate target product is determined based on the multi-dimensional exposure data and consumption data of each candidate target product in the database, including: If the cumulative exposure of the product dimension of the candidate target product is greater than or equal to the effective exposure threshold of the product dimension, then the value of the candidate target product is determined based on the cumulative consumption and cumulative exposure of the product dimension. If the cumulative exposure of the candidate target product in the product dimension is less than the effective exposure threshold of the product dimension, and the cumulative exposure of the candidate target product in the advertiser dimension is greater than or equal to the effective exposure threshold of the candidate target product in the advertiser dimension, then the value of the candidate target product is determined based on the cumulative consumption of the candidate target product in the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, and the cumulative exposure of the product dimension. If the cumulative exposure of the candidate target product in the product dimension is greater than or equal to the effective exposure threshold of the product dimension, the cumulative exposure of the candidate target product in the advertiser dimension is less than the effective exposure threshold of the advertiser dimension, and the cumulative exposure of the candidate target product in the product placement dimension is greater than or equal to the effective exposure threshold of the product placement dimension, then the value of the candidate target product is determined based on the cumulative consumption of the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, the cumulative exposure of the product dimension, the cumulative consumption of the product placement dimension, and the cumulative exposure of the product placement dimension.

[0068] Typically, product value is assessed solely from the product perspective. If a product's daily exposure and actual consumption are too low, the cumulative value of the product will fluctuate significantly, leading to inaccurate value calculations.

[0069] Based on this, this solution takes into account both the accuracy and reliability of the statistics and adopts a product value smoothing estimation method that integrates multi-level exposure data. The algorithm achieves robust estimation of the cumulative product value in sparse exposure scenarios through hierarchical compensation and weighted fusion based on three levels of granularity: product dimension, advertiser dimension, and advertiser location dimension.

[0070] Figure 4 This is a schematic diagram illustrating the overall framework of a value smoothing update method provided in this application embodiment. By querying the list of all ad orders in the database, ad order dimension data, advertiser temperature data, and ad placement dimension data can be obtained. Then, the cumulative value corresponding to each ad order is calculated using a smoothing estimation method, and the field information used to record the cumulative value of the ad order is updated. Subsequently, the latest cumulative value of the ad order is read in real time from this field information.

[0071] Figure 5This is a flowchart illustrating a smoothing estimation algorithm provided in an embodiment of this application. By performing layer-by-layer smoothing calculations on data across three dimensions, the final cumulative value of an advertisement can be obtained. The following example can be used to illustrate this. Figure 5 To understand.

[0072] The calculation process of cumulative eCPM (a parameter that measures the value of an advertising unit) is illustrated below: Step 1: Obtain ,like ≥ ,but

[0073] Step 2: If < and ≥ ,but ) / ) Step 3: If the conditions of Step 2 are not yet met, and ≥ ,but ) / ) If none of the above three conditions are met, the data will be marked as invalid and will be re-evaluated in subsequent batches.

[0074] in, This represents the cumulative impressions of an ad within a single dimension. This represents the cumulative impressions from the advertiser's perspective. This indicates the cumulative impressions for each ad placement. This indicates the cumulative cost per ad segment. This represents the cumulative spending by advertiser. This indicates the cumulative cost per ad slot. This represents the minimum effective exposure for a single ad dimension. This represents the minimum effective exposure for the advertiser. This indicates the minimum effective exposure for an ad placement. Indicates advertiser-level weight. This indicates the ad placement weight.

[0075] Figure 6 Flowchart of the data processing method provided in the embodiments of this application Figure 3 In step S103, based on the real-time operating status data of the bidding platform and the queue length control strategy, the control length of the original recall queue is determined, including: S301. Obtain real-time operating status data of the bidding platform according to the preset time window.

[0076] Operational status data includes: timeout rate or anomaly rate.

[0077] To achieve dynamic and flexible control of the recall queue, this solution introduces an adaptive and dynamic adjustment of the recall queue length based on the real-time operating status of the system (i.e., the bidding platform) to ensure that the system remains stable and efficient under high load.

[0078] Each node instance of the bidding platform maintains a scheduled task to monitor and report the platform's operational status data in real time. In this embodiment, the operational status data may include timeout rate and anomaly rate.

[0079] It can periodically obtain real-time operational status data of the bidding platform according to a preset time window.

[0080] S302. Based on the real-time operating status data of the bidding platform during the current time window and the preset threshold indicators, determine the current control method of the original recall queue.

[0081] The control methods include: reducing the queue length or increasing the queue length.

[0082] By comparing real-time operational status data with preset threshold indicators, it can be determined whether to adjust the length of the original recall queue. Typically, when the platform is under high load, the length of the original recall queue can be truncated in a certain step ratio, i.e., the queue length is reduced, to quickly alleviate platform pressure. Conversely, when the platform gradually returns to stable operation, the length of the original recall queue can be restored in a certain step ratio, i.e., the queue length is increased.

[0083] It is worth noting that both increasing and decreasing the queue length here involve truncating a certain length from the original recall queue. However, when the queue length is increased, the truncation of the original recall queue length becomes less and less.

[0084] For example: If the original recall queue length is 1000, in the first time window, the queue length is reduced by 100, so the original recall queue length becomes 900; in the second time window, the queue length is reduced by 80, so the original recall queue length becomes 820; in the third time window, the queue length is increased by 50, so the original recall queue length becomes 870. Although the queue length is increased, it still requires 130 units to be truncated from the original recall queue length of 1000.

[0085] S303. Determine the current control length of the original recall queue based on the current control mode, the first control ratio corresponding to the control mode, and the length of the recall queue after processing in the previous time window.

[0086] In this embodiment, different control methods correspond to control ratios. The length of the recall queue after the previous time window is calculated based on the determined current control method and the first control ratio corresponding to that control method, so as to obtain the current control length of the original recall queue.

[0087] In other words, the calculation of the adjustment length under each time window is based on the length of the recall queue after the adjustment in the previous time window.

[0088] For example, the original recall queue length is 1000, and the recall queue length after the first time window is 900. Assuming the current time window is the second time window, and the current adjustment method is to reduce the queue length, the first adjustment ratio corresponding to this adjustment method is 10%, that is, to reduce it by 10%, then 900 10% = 90, which means the second time window. The length of the recall queue needs to be reduced by 90 from 900, that is, the length of the original recall queue should be adjusted to 810. Therefore, the current adjusted length of the original recall queue is determined to be 190, which means that 190 data points need to be truncated from the original recall queue, leaving 810 data points.

[0089] Optionally, in step S302, based on the real-time operating status data of the bidding platform within the current time window and preset threshold indicators, the current control method for the original recall queue is determined, including: If the timeout rate or anomaly rate of the bidding platform is greater than the preset threshold, the current control method for the original recall queue is to reduce the queue length; if the timeout rate and anomaly rate of the bidding platform are less than or equal to the preset threshold, the current control method for the original recall queue is to increase the queue length.

[0090] In some embodiments, the preset threshold can be 3%. When the timeout rate or anomaly rate of the bidding platform is determined to be greater than 3%, it can be determined that the platform is under great processing pressure, and the current control method can be determined to be to reduce the queue length; while when the timeout rate and anomaly rate of the bidding platform are both less than or equal to 3%, the system pressure can be relieved, and a certain length can be gradually restored, and the current control method can be determined to be to increase the queue length.

[0091] Figure 7 Flowchart of the data processing method provided in the embodiments of this application Figure 4 Optionally, in step S302, the current control length of the original recall queue is determined based on the current control mode, the control ratio corresponding to the control mode, and the length of the recall queue after processing in the previous time window, including: S401. Determine the target length based on the length of the recall queue after processing in the previous time window and the corresponding adjustment ratio of the adjustment method.

[0092] The length of the recall queue after processing in the previous time window refers to the original recall queue length that should be retained in the previous time window. Assuming the original recall queue length is 1000, and the recall queue length after processing in the previous time window is 900 (meaning the original recall queue length after truncation in the previous time window is 900), and the corresponding adjustment ratio is 10%, then the target length can be achieved through 900. The calculation yields a target length of 90, calculated as 10%.

[0093] S402. Calculate the current control length of the original recall queue based on the current control method, the length of the recall queue after the previous time window processing, and the target length.

[0094] Assuming the current control method is to reduce the queue length, then in the current time window, the length needs to be reduced by 90 from 900, that is, retaining a length of 810. Since the original recall queue length is 1000, the current control length of the original recall queue is 1000 minus 810, that is, the original recall queue needs to be truncated by 190, retaining a length of 810.

[0095] Optionally, the control length here can be defined as either a cutoff length or a retention length. When it is a cutoff length, the current control length is determined to be 190, as shown in the example. When it is a retention length, the current control length is determined to be 810.

[0096] Optionally, the method further includes: if the timeout rate and anomaly rate of the bidding platform obtained in multiple consecutive time windows are all less than or equal to the preset threshold indicators, then the current control length of the original recall queue is determined step by step according to the second control ratio until the length of the controlled queue reaches the length of the original recall queue.

[0097] In some embodiments, if the timeout rate and anomaly rate of the bidding platform remain less than or equal to the preset threshold indicators after multiple consecutive time windows, it can be considered that the platform has gradually resumed stable operation. At this time, the current control length of the original recall queue can be determined step by step according to the second control ratio.

[0098] It's worth noting that when the control method is to reduce the queue length, the first control ratio can be 10%, meaning a reduction in steps of 10%. When the control method is to increase the queue length, the second control ratio can be 5%, meaning an increase in steps of 5%. The reason for setting a relatively small increase ratio is that the platform is gradually stabilizing, and if the queue length recovers too quickly, it may put pressure on the platform again. When determining the current control length of the original recall queue step by step according to the second control ratio, since the queue length is being increased according to the second control ratio, this second control ratio can be greater than 5%. That is, after multiple time windows, if the platform remains stable, the recovery of the queue length can be accelerated at a larger rate. For example, the second control ratio can be 10%.

[0099] Once the platform's timeout and anomaly rates return to normal, the platform records the recovery time. After three time windows, if the platform is running smoothly, the queue length is gradually restored in 10% increments until it is fully restored to the initial length of the original recall queue or a new dynamic balance is reached.

[0100] Optionally, the method further includes: if the length of the adjusted queue indicated by the current adjustment length of the original recall queue reaches a preset adjustment threshold, then the adjustment is stopped.

[0101] In some embodiments, whether the queue length is increased or decreased, a preset control threshold can be set to control the length of the original recall queue within a reasonable range, avoiding unlimited decreases or increases.

[0102] In other words, during the process of reducing the queue length, when the current control length of the original recall queue has reached the lower limit, even if the platform's timeout rate or anomaly rate is still greater than 3%, the length of the original recall queue will not be truncated again. Similarly, during the process of increasing the queue length, when the current control length of the original recall queue has reached the upper limit, even if the platform is running stably, the length of the original recall queue will not be restored.

[0103] Figure 8 This is a schematic diagram of a dynamic queue length control process provided in an embodiment of this application. Figure 8 The specific steps have been explained in detail above and will not be repeated here.

[0104] By using the dynamic queue length control mechanism in this solution, the queue length can be effectively controlled and responded to in case of sudden traffic surges. After the pressure is relieved, the queue length can be restored smoothly, which can avoid the long-term impact of excessive truncation on product exposure opportunities.

[0105] Moreover, this solution possesses significant traffic adaptability. During off-peak hours, platform traffic is low, the recall queue is short, and competition is not intense. At this time, queue truncation is minimized, or even eliminated, to maximize product exposure opportunities and improve product campaign performance. During peak hours, traffic surges, and product competition intensifies. At this time, the truncation ratio is dynamically increased based on real-time timeout and anomaly rates, effectively protecting the overall stability and performance of the platform and preventing overload.

[0106] Figure 9 Flowchart of the data processing method provided in the embodiments of this application Figure 5 Optionally, in step S104, the original recall queue is truncated according to the adjusted length and a preset truncation strategy to obtain the target recall queue, including: S501. Determine the truncation length of the original recall queue based on the length of the original recall queue and the adjustment length.

[0107] In some embodiments, the calculated control length can itself be used as the truncation length. In other embodiments, when the control length is the length that needs to be retained, the control length can be subtracted from the length of the original recall queue to obtain the truncation length. Whether the control length is set as the truncation length or the retained length can be flexibly adjusted. The sum of the truncation length and the retained length is the length of the original recall queue.

[0108] S502. Based on the distributor to which each candidate target product belongs in the original recall queue, group each candidate target product into sub-queues corresponding to each distributor.

[0109] Taking an ad order as an example, in this embodiment, ad orders can be grouped according to the advertiser to which each ad order belongs in the original recall queue, and the ad orders can be divided into several sub-queues. Each sub-queue represents the bidding list of an advertiser. That is, ad orders belonging to the same advertiser in the original recall queue are divided into one sub-queue.

[0110] S503. Determine the product truncation length for each advertiser based on the data length of the sub-queue corresponding to each advertiser and the truncation length of the original recall queue.

[0111] The length ratio of each advertiser's sub-queue can be determined based on the length of each advertiser's sub-queue and the length of the original recall queue; then, the product truncation length for each advertiser can be determined based on the length ratio of each advertiser's sub-queue and the truncation length of the original recall queue.

[0112] For example: Assume the original recall queue has a length of 1000, the truncation length is 400, that is, the length to be retained is 600. Also assume that the lengths of the sub-queues corresponding to advertisers A, B, and C are 40% (400), 40% (400), and 20% (200) of the original recall queue length, respectively.

[0113] Therefore, the truncated length for advertiser A is 600. 40% = 240; The truncated length for advertiser B is 600. 40% = 240; the truncated length for advertiser C is 600. 20% = 120. Therefore, the cutoff length for advertiser A is 400 - 240 = 160; the cutoff length for advertiser B is 400 - 240 = 160; and the cutoff length for advertiser C is 200 - 120 = 80.

[0114] S504. Based on the product truncation length corresponding to each advertiser, truncate each candidate target product in the sub-queue corresponding to each advertiser to obtain the retained product information corresponding to each advertiser.

[0115] Based on the truncation length corresponding to each advertiser's subqueue, a corresponding number of products can be truncated from each advertiser's subqueue to obtain the retained product information for each advertiser. In other words, the untruncated ad order information retained by each advertiser after subqueue truncation is obtained.

[0116] S505. Based on the retained product information of each advertiser, obtain the target recall queue.

[0117] By using the retained ad order information for each advertiser, a target recall queue can be obtained. The target recall queue contains the ad orders selected after the initial screening using this solution to participate in the subsequent fine screening.

[0118] Of course, in other product scenarios, the target recall queue contains product information of other products that have been filtered out.

[0119] By using this method for truncation, we can ensure that the product distribution of the target recall queue after truncation is highly consistent with that of the original recall queue, effectively avoiding excessive pressure on specific advertisers and maintaining the healthy diversity of the product ecosystem.

[0120] Traditional truncation methods, which prioritize products based solely on value, often result in lower-ranked products being truncated. This can lead to smaller advertisers with fewer ads and less product exposure having lower-value products ranked lower and thus being prioritized for truncation. This solution, however, groups products by advertiser, ensuring that each advertiser retains a certain percentage of their products. This prevents high-value products from smaller advertisers from being mistakenly truncated, maintaining fairness while maximizing the exposure of high-value products.

[0121] Figure 10 Flowchart of the data processing method provided in the embodiments of this application Figure 6 Optionally, in step S504, based on the product truncation length corresponding to each advertiser, the candidate target products in the sub-queue corresponding to each advertiser are truncated to obtain the retained product information corresponding to each advertiser, including: S601. Based on the value of each candidate target product in the sub-queue corresponding to the advertiser, sort the candidate target products to obtain the target sub-queue corresponding to the advertiser.

[0122] In some embodiments, before performing internal truncation on the sub-queues corresponding to each advertiser, the value of each candidate target product in each advertiser's sub-queue can be sorted based on the grouping results.

[0123] Continuing with the example of ad orders, we can sort the ad orders in each advertiser's sub-queue by value.

[0124] S602. Determine the first retention quantity based on the length of the sub-queue corresponding to the delivery party and the preset retention ratio.

[0125] In some embodiments, the internal truncation process for advertisers' sub-queues can prioritize retaining the top 30% of high-value ad orders to ensure that overall advertising costs, conversions, and revenue are not affected to the greatest extent possible, thus determining the platform's core revenue. Then, from the remaining 70% of ad orders, a random shuffling algorithm is used to select the remaining ad orders to ensure that non-top ad orders (small advertisers) have an equal probability of winning when entering the recall queue, effectively avoiding the crowding-out effect caused by fixed sorting and improving the fairness of the advertising ecosystem.

[0126] For example, assume that the length of the subqueue corresponding to the advertiser is 400, the truncation length is 180, and the retention length is 220.

[0127] Assuming the preset retention rate is 30%, the first retention quantity is: 400. 30%, which is 120.

[0128] S603. Based on the first reserved quantity, select a plurality of first target products from the target sub-queue corresponding to the delivery party according to the sorting result of each candidate target product.

[0129] Taking the value of each ad in the advertiser's sub-queue as sorted from high to low as an example, the top 120 ad in order can be selected as the first number of ad in the retention list.

[0130] S604. Based on the product cut-off length corresponding to the advertiser and the first retention quantity, determine the second retention quantity, and based on the second retention quantity, select multiple second target products from the remaining candidate target products in the target sub-queue corresponding to the advertiser according to a random algorithm.

[0131] The retention length can be directly determined based on the truncation length. As mentioned above, the retention length is 220. Since 120 ad orders have already been selected according to the first ratio, the second retention quantity is 100. These 100 ad orders can be randomly selected from the remaining 280 ad orders in the sub-queue using a random shuffling algorithm.

[0132] S605, the retained product information corresponding to the distributor includes a first number of multiple first target products and a second number of multiple second target products.

[0133] Ultimately, the 220 selected ad orders were used as the retained product information for the advertiser.

[0134] In practical applications, the preset retention ratio can also be dynamically adjusted according to the actual situation. In short, when the sub-queue of the advertiser is internally truncated, as many high-value products as possible should be retained, while a certain proportion should be reserved to screen from low-value products, so as to allocate an equal retention probability to all products.

[0135] Figure 11 This is a schematic diagram of a truncation processing flow provided in an embodiment of this application. Figure 11 The implementation of each step has been explained in detail above and will not be repeated here.

[0136] Optionally, in step S505, the target recall queue is obtained based on the reserved product information corresponding to each advertiser, including: using the reserved product information corresponding to each advertiser as product information in the target recall queue to obtain the target recall queue.

[0137] By merging the retained products from each advertiser, a target recall queue can be obtained. During merging, it is not necessary to sort by value; random merging is acceptable. The purpose of this solution is to roughly filter out the target recall queue. The platform can then further refine the selection based on this queue to choose the final products for advertising.

[0138] In summary, the data processing method provided in this embodiment includes: determining the value of each candidate target product based on the data of each candidate target product recorded in the database; determining an original recall queue based on the value of each candidate target product; the original recall queue includes multiple candidate target products, which are sorted according to their value; determining the adjustment length of the original recall queue based on the real-time operating status data of the bidding platform and the queue length adjustment strategy; and truncating the original recall queue according to the adjustment length and a preset truncation strategy to obtain the target recall queue. This method dynamically adjusts the queue length of the original recall queue based on the real-time operating status data of the bidding platform, which can adapt to different traffic scenarios, improve resource utilization, avoid performance bottlenecks of the bidding platform, and ensure the stable operation and rapid response of the platform.

[0139] Secondly, when calculating product value, smoothing the calculation by integrating multi-dimensional data can improve the accuracy of product value calculation and the precision of subsequent truncation processing.

[0140] In addition, when performing truncation, products are grouped by advertisers, and then truncation is carried out by combining value ranking, prioritizing the retention of high-value products, and randomly shuffling low-value products. This ensures fair exposure of advertisers' products while protecting the overall revenue and commercial value of the platform, minimizing the loss of high-value products, and promoting product diversity.

[0141] The following describes the apparatus, equipment, and storage medium used to execute the data processing method provided in this application. The specific implementation process and technical effects are described above and will not be repeated here.

[0142] Figure 12 This is a schematic diagram of a data processing apparatus provided in an embodiment of this application. The functions implemented by this data processing apparatus correspond to the steps performed by the method described above. Figure 12 As shown, the device includes: a determining module 100 and a processing module 200; The determination module 100 is used to determine the value of each candidate target product based on the data of each candidate target product recorded in the database; The determination module 100 is used to determine the original recall queue based on the value of each candidate target product; the original recall queue includes multiple candidate target products, and each candidate target product is sorted according to its value. The determination module 100 is used to determine the adjustment length of the original recall queue based on the real-time operating status data of the bidding platform and the queue length adjustment strategy. The processing module 200 is used to truncate the original recall queue according to the controlled length and the preset truncation strategy to obtain the target recall queue.

[0143] Optionally, module 100 is specifically used to determine the advertiser and product placement position of each candidate target product based on the identifier of each candidate target product. Retrieve multi-dimensional exposure and consumption data for each candidate target product from the database; The value of each candidate target product is determined based on its multi-dimensional exposure and consumption data. The multi-dimensional data includes one or more of the following: product dimension, advertiser dimension, and product placement dimension.

[0144] Optionally, the determining module 100 is specifically used to determine the value of the candidate target product based on the cumulative consumption and cumulative exposure of the product dimension if the cumulative exposure of the product dimension of the candidate target product is greater than or equal to the effective exposure threshold of the product dimension. If the cumulative exposure of the candidate target product in the product dimension is less than the effective exposure threshold of the product dimension, and the cumulative exposure of the candidate target product in the advertiser dimension is greater than or equal to the effective exposure threshold of the candidate target product in the advertiser dimension, then the value of the candidate target product is determined based on the cumulative consumption of the candidate target product in the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, and the cumulative exposure of the product dimension. If the cumulative exposure of the candidate target product in the product dimension is greater than or equal to the effective exposure threshold of the product dimension, the cumulative exposure of the candidate target product in the advertiser dimension is less than the effective exposure threshold of the advertiser dimension, and the cumulative exposure of the candidate target product in the product placement dimension is greater than or equal to the effective exposure threshold of the product placement dimension, then the value of the candidate target product is determined based on the cumulative consumption of the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, the cumulative exposure of the product dimension, the cumulative consumption of the product placement dimension, and the cumulative exposure of the product placement dimension.

[0145] Optionally, module 100 is specifically used to obtain real-time operating status data of the bidding platform according to a preset time window. The operating status data includes: timeout rate and anomaly rate. Based on the real-time operating status data of the bidding platform during the current time window and the preset threshold indicators, the current adjustment method of the original recall queue is determined. The adjustment method includes: reducing the queue length or increasing the queue length. The current control length of the original recall queue is determined based on the current control method, the first control ratio corresponding to the control method, and the length of the recall queue after processing in the previous time window.

[0146] Optionally, module 100 is specifically used to determine that if the timeout rate or anomaly rate of the bidding platform is greater than a preset threshold indicator, the current control method of the original recall queue is to reduce the queue length. If the timeout rate and anomaly rate of the bidding platform are less than or equal to the preset threshold indicators, then the current control method for the original recall queue is to increase the queue length.

[0147] Optionally, the determining module 100 is specifically used to determine the target length based on the length of the recall queue after processing in the previous time window and the control ratio corresponding to the control method; The current control length of the original recall queue is calculated based on the current control method, the length of the recall queue after processing in the previous time window, and the target length.

[0148] Optionally, the processing module 200 is further configured to determine the current control length of the original recall queue step by step according to the second control ratio if the timeout rate and abnormal rate of the bidding platform obtained in multiple consecutive time windows are all less than or equal to the preset threshold indicators, until the length of the controlled queue reaches the length of the original recall queue.

[0149] Optionally, the processing module 200 is further configured to stop the regulation if the length of the regulated queue indicated by the current regulation length of the original recall queue reaches a preset regulation threshold.

[0150] Optionally, the processing module 200 is specifically used to determine the truncation length of the original recall queue based on the length of the original recall queue and the adjustment length. Based on the distributor to which each candidate target product belongs in the original recall queue, the candidate target products are grouped to obtain the sub-queues corresponding to each distributor. The product truncation length for each advertiser is determined based on the data length of the sub-queue corresponding to each advertiser and the truncation length of the original recall queue. Based on the product truncation length corresponding to each advertiser, the candidate target products in the sub-queue corresponding to each advertiser are truncated to obtain the retained product information corresponding to each advertiser. Based on the retained product information of each advertiser, a target recall queue is obtained.

[0151] Optionally, the processing module 200 is specifically used to sort each candidate target product according to the value of each candidate target product in the sub-queue corresponding to the advertiser, so as to obtain the target sub-queue corresponding to the advertiser. The first retention quantity is determined based on the length of the sub-queue corresponding to the delivery party and the preset retention ratio; Based on the first retention quantity, select multiple first target products from the target sub-queue corresponding to the advertiser according to the sorting results of each candidate target product; Based on the product truncation length corresponding to the advertiser and the first retention quantity, the second retention quantity is determined, and based on the second retention quantity, multiple second target products are selected from the remaining candidate target products in the target sub-queue corresponding to the advertiser according to a random algorithm. The retained product information corresponding to the advertiser includes a first number of multiple first target products and a second number of multiple second target products.

[0152] Optionally, the processing module 200 is specifically used to use the reserved product information corresponding to each advertiser as product information in the target recall queue to obtain the target recall queue.

[0153] These modules can be one or more integrated circuits configured to implement the above methods, such as one or more Application Specific Integrated Circuits (ASICs), one or more digital signal processors (DSPs), or one or more Field Programmable Gate Arrays (FPGAs). Alternatively, when a module is implemented using processing element scheduler code, the processing element can be a general-purpose processor, such as a Central Processing Unit (CPU) or other processor capable of calling program code. Furthermore, these modules can be integrated together as a system-on-a-chip (SOC).

[0154] The modules described above can be connected or communicate with each other via wired or wireless connections. Wired connections can include metal cables, optical fibers, hybrid cables, or any combination thereof. Wireless connections can include connections via LAN, WAN, Bluetooth, ZigBee, or NFC, or any combination thereof. Two or more modules can be combined into a single module, and any module can be divided into two or more units. Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here.

[0155] Figure 13 A schematic diagram of an electronic device provided in this application embodiment includes: a processor 801, a storage medium 802, and a bus 803. The storage medium 802 stores machine-readable instructions executable by the processor 801. When the electronic device runs a data processing method as described in the embodiment, the processor 801 communicates with the storage medium 802 via the bus 803. The processor 801 executes the machine-readable instructions to perform the following steps: The value of each candidate target product is determined based on the data of each candidate target product recorded in the database; The original recall queue is determined based on the value of each candidate target product; the original recall queue includes multiple candidate target products, and each candidate target product is sorted according to its value. The adjustment length of the original recall queue is determined based on the real-time operating status data of the bidding platform and the queue length adjustment strategy. Based on the controlled length and the preset truncation strategy, the original recall queue is truncated to obtain the target recall queue.

[0156] In one feasible implementation, when the processor 801 determines the value of each candidate target product based on the data of each candidate target product recorded in the database, it is specifically used to: determine the delivery party and product delivery position of each candidate target product based on the identifier of each candidate target product; Retrieve multi-dimensional exposure and consumption data for each candidate target product from the database; The value of each candidate target product is determined based on its multi-dimensional exposure and consumption data. The multi-dimensional data includes one or more of the following: product dimension, advertiser dimension, and product placement dimension.

[0157] In a feasible implementation, when the processor 801 determines the value of each candidate target product based on the multi-dimensional exposure data and consumption data of each candidate target product in the database, it specifically performs the following: if the cumulative exposure of the product dimension of the candidate target product is greater than or equal to the effective exposure threshold of the product dimension, then the value of the candidate target product is determined based on the cumulative consumption of the product dimension and the cumulative exposure of the product dimension. If the cumulative exposure of the candidate target product in the product dimension is less than the effective exposure threshold of the product dimension, and the cumulative exposure of the candidate target product in the advertiser dimension is greater than or equal to the effective exposure threshold of the candidate target product in the advertiser dimension, then the value of the candidate target product is determined based on the cumulative consumption of the candidate target product in the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, and the cumulative exposure of the product dimension. If the cumulative exposure of the candidate target product in the product dimension is greater than or equal to the effective exposure threshold of the product dimension, the cumulative exposure of the candidate target product in the advertiser dimension is less than the effective exposure threshold of the advertiser dimension, and the cumulative exposure of the candidate target product in the product placement dimension is greater than or equal to the effective exposure threshold of the product placement dimension, then the value of the candidate target product is determined based on the cumulative consumption of the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, the cumulative exposure of the product dimension, the cumulative consumption of the product placement dimension, and the cumulative exposure of the product placement dimension.

[0158] In a feasible implementation, when the processor 801 determines the adjustment length of the original recall queue based on the real-time operating status data of the bidding platform and the queue length adjustment strategy, it is specifically used to: obtain the real-time operating status data of the bidding platform according to a preset time window, the operating status data including: timeout rate and error rate; Based on the real-time operating status data of the bidding platform during the current time window and the preset threshold indicators, the current adjustment method of the original recall queue is determined. The adjustment method includes: reducing the queue length or increasing the queue length. The current control length of the original recall queue is determined based on the current control method, the first control ratio corresponding to the control method, and the length of the recall queue after processing in the previous time window.

[0159] In a feasible implementation, when the processor 801 determines the current control method of the original recall queue based on the real-time running status data of the bidding platform in the current time window and the preset threshold indicators, it is specifically used to: if the timeout rate or abnormality rate of the bidding platform is greater than the preset threshold indicators, then determine that the current control method of the original recall queue is to reduce the queue length. If the timeout rate and anomaly rate of the bidding platform are less than or equal to the preset threshold indicators, then the current control method for the original recall queue is to increase the queue length.

[0160] In a feasible implementation, when the processor 801 determines the current control length of the original recall queue based on the current control method, the control ratio corresponding to the control method, and the length of the recall queue after the previous time window, it is specifically used to: determine the target length based on the length of the recall queue after the previous time window and the control ratio corresponding to the control method. The current control length of the original recall queue is calculated based on the current control method, the length of the recall queue after processing in the previous time window, and the target length.

[0161] In a feasible implementation, the processor 801 is also used to perform the following: if the timeout rate and anomaly rate of the bidding platform obtained in multiple consecutive time windows are all less than or equal to the preset threshold indicators, then determine the current adjustment length of the original recall queue step by step according to the second adjustment ratio, until the length of the adjusted queue reaches the length of the original recall queue.

[0162] In one feasible implementation, the processor 801 is further configured to: stop regulation if the length of the regulated queue indicated by the current regulation length of the original recall queue reaches a preset regulation threshold.

[0163] In one feasible implementation, when the processor 801 performs truncation processing on the original recall queue according to the control length and the preset truncation strategy to obtain the target recall queue, it is specifically used to: determine the truncation length of the original recall queue according to the length of the original recall queue and the control length. Based on the distributor to which each candidate target product belongs in the original recall queue, the candidate target products are grouped to obtain the sub-queues corresponding to each distributor. The product truncation length for each advertiser is determined based on the data length of the sub-queue corresponding to each advertiser and the truncation length of the original recall queue. Based on the product truncation length corresponding to each advertiser, the candidate target products in the sub-queue corresponding to each advertiser are truncated to obtain the retained product information corresponding to each advertiser. Based on the retained product information of each advertiser, a target recall queue is obtained.

[0164] In a feasible implementation, when the processor 801 performs truncation processing on each candidate target product in the sub-queue corresponding to each advertiser according to the product truncation length corresponding to each advertiser, and obtains the retained product information corresponding to each advertiser, it is specifically used to: sort each candidate target product according to the value of each candidate target product in the sub-queue corresponding to the advertiser, and obtain the target sub-queue corresponding to the advertiser. The first retention quantity is determined based on the length of the sub-queue corresponding to the delivery party and the preset retention ratio; Based on the first retention quantity, select multiple first target products from the target sub-queue corresponding to the advertiser according to the sorting results of each candidate target product; Based on the product truncation length corresponding to the advertiser and the first retention quantity, the second retention quantity is determined, and based on the second retention quantity, multiple second target products are selected from the remaining candidate target products in the target sub-queue corresponding to the advertiser according to a random algorithm. The retained product information corresponding to the advertiser includes a first number of multiple first target products and a second number of multiple second target products.

[0165] In a feasible implementation, when the processor 801 executes the process of obtaining the target recall queue based on the reserved product information corresponding to each distributor, it specifically uses the reserved product information corresponding to each distributor as the product information in the target recall queue to obtain the target recall queue.

[0166] The storage medium 802 stores program code, which, when executed by the processor 801, causes the processor 801 to perform various steps in the data processing methods according to various exemplary embodiments of this application as described in the "Exemplary Methods" section above.

[0167] The processor 801 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0168] Storage medium 802, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. Storage medium 802 in the embodiments of this application can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0169] Optionally, embodiments of this application also provide a computer-readable storage medium storing a computer program, which is executed by a processor, and the processor performs the following steps: The value of each candidate target product is determined based on the data of each candidate target product recorded in the database; The original recall queue is determined based on the value of each candidate target product; the original recall queue includes multiple candidate target products, and each candidate target product is sorted according to its value. The adjustment length of the original recall queue is determined based on the real-time operating status data of the bidding platform and the queue length adjustment strategy. Based on the controlled length and the preset truncation strategy, the original recall queue is truncated to obtain the target recall queue.

[0170] In one feasible implementation, when the processor 801 determines the value of each candidate target product based on the data of each candidate target product recorded in the database, it is specifically used to: determine the delivery party and product delivery position of each candidate target product based on the identifier of each candidate target product; Retrieve multi-dimensional exposure and consumption data for each candidate target product from the database; The value of each candidate target product is determined based on its multi-dimensional exposure and consumption data. The multi-dimensional data includes one or more of the following: product dimension, advertiser dimension, and product placement dimension.

[0171] In a feasible implementation, when the processor 801 determines the value of each candidate target product based on the multi-dimensional exposure data and consumption data of each candidate target product in the database, it specifically performs the following: if the cumulative exposure of the product dimension of the candidate target product is greater than or equal to the effective exposure threshold of the product dimension, then the value of the candidate target product is determined based on the cumulative consumption of the product dimension and the cumulative exposure of the product dimension. If the cumulative exposure of the candidate target product in the product dimension is less than the effective exposure threshold of the product dimension, and the cumulative exposure of the candidate target product in the advertiser dimension is greater than or equal to the effective exposure threshold of the candidate target product in the advertiser dimension, then the value of the candidate target product is determined based on the cumulative consumption of the candidate target product in the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, and the cumulative exposure of the product dimension. If the cumulative exposure of the candidate target product in the product dimension is greater than or equal to the effective exposure threshold of the product dimension, the cumulative exposure of the candidate target product in the advertiser dimension is less than the effective exposure threshold of the advertiser dimension, and the cumulative exposure of the candidate target product in the product placement dimension is greater than or equal to the effective exposure threshold of the product placement dimension, then the value of the candidate target product is determined based on the cumulative consumption of the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, the cumulative exposure of the product dimension, the cumulative consumption of the product placement dimension, and the cumulative exposure of the product placement dimension.

[0172] In a feasible implementation, when the processor 801 determines the adjustment length of the original recall queue based on the real-time operating status data of the bidding platform and the queue length adjustment strategy, it is specifically used to: obtain the real-time operating status data of the bidding platform according to a preset time window, the operating status data including: timeout rate and error rate; Based on the real-time operating status data of the bidding platform during the current time window and the preset threshold indicators, the current adjustment method of the original recall queue is determined. The adjustment method includes: reducing the queue length or increasing the queue length. The current control length of the original recall queue is determined based on the current control method, the first control ratio corresponding to the control method, and the length of the recall queue after processing in the previous time window.

[0173] In a feasible implementation, when the processor 801 determines the current control method of the original recall queue based on the real-time running status data of the bidding platform in the current time window and the preset threshold indicators, it is specifically used to: if the timeout rate or abnormality rate of the bidding platform is greater than the preset threshold indicators, then determine that the current control method of the original recall queue is to reduce the queue length. If the timeout rate and anomaly rate of the bidding platform are less than or equal to the preset threshold indicators, then the current control method for the original recall queue is to increase the queue length.

[0174] In a feasible implementation, when the processor 801 determines the current control length of the original recall queue based on the current control method, the control ratio corresponding to the control method, and the length of the recall queue after the previous time window, it is specifically used to: determine the target length based on the length of the recall queue after the previous time window and the control ratio corresponding to the control method. The current control length of the original recall queue is calculated based on the current control method, the length of the recall queue after processing in the previous time window, and the target length.

[0175] In a feasible implementation, the processor 801 is also used to perform the following: if the timeout rate and anomaly rate of the bidding platform obtained in multiple consecutive time windows are all less than or equal to the preset threshold indicators, then determine the current adjustment length of the original recall queue step by step according to the second adjustment ratio, until the length of the adjusted queue reaches the length of the original recall queue.

[0176] In one feasible implementation, the processor 801 is further configured to: stop regulation if the length of the regulated queue indicated by the current regulation length of the original recall queue reaches a preset regulation threshold.

[0177] In one feasible implementation, when the processor 801 performs truncation processing on the original recall queue according to the control length and the preset truncation strategy to obtain the target recall queue, it is specifically used to: determine the truncation length of the original recall queue according to the length of the original recall queue and the control length. Based on the distributor to which each candidate target product belongs in the original recall queue, the candidate target products are grouped to obtain the sub-queues corresponding to each distributor. The product truncation length for each advertiser is determined based on the data length of the sub-queue corresponding to each advertiser and the truncation length of the original recall queue. Based on the product truncation length corresponding to each advertiser, the candidate target products in the sub-queue corresponding to each advertiser are truncated to obtain the retained product information corresponding to each advertiser. Based on the retained product information of each advertiser, a target recall queue is obtained.

[0178] In a feasible implementation, when the processor 801 performs truncation processing on each candidate target product in the sub-queue corresponding to each advertiser according to the product truncation length corresponding to each advertiser, and obtains the retained product information corresponding to each advertiser, it is specifically used to: sort each candidate target product according to the value of each candidate target product in the sub-queue corresponding to the advertiser, and obtain the target sub-queue corresponding to the advertiser. The first retention quantity is determined based on the length of the sub-queue corresponding to the delivery party and the preset retention ratio; Based on the first retention quantity, select multiple first target products from the target sub-queue corresponding to the advertiser according to the sorting results of each candidate target product; Based on the product truncation length corresponding to the advertiser and the first retention quantity, the second retention quantity is determined, and based on the second retention quantity, multiple second target products are selected from the remaining candidate target products in the target sub-queue corresponding to the advertiser according to a random algorithm. The retained product information corresponding to the advertiser includes a first number of multiple first target products and a second number of multiple second target products.

[0179] In a feasible implementation, when the processor 801 executes the process of obtaining the target recall queue based on the reserved product information corresponding to each distributor, it specifically uses the reserved product information corresponding to each distributor as the product information in the target recall queue to obtain the target recall queue.

[0180] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0184] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data processing method, characterized in that, include: The value of each candidate target product is determined based on the data of each candidate target product recorded in the database; The original recall queue is determined based on the value of each candidate target product; the original recall queue includes multiple candidate target products, and each candidate target product is sorted according to its value. The adjustment length of the original recall queue is determined based on the real-time operating status data of the bidding platform and the queue length adjustment strategy. Based on the adjusted length and the preset truncation strategy, the original recall queue is truncated to obtain the target recall queue.

2. The method according to claim 1, characterized in that, The step of determining the value of each candidate target product based on the data of each candidate target product recorded in the database includes: Based on the identifiers of each candidate target product, determine the advertiser and product placement for each candidate target product; Retrieve multi-dimensional exposure and consumption data for each candidate target product from the database; The value of each candidate target product is determined based on its multi-dimensional exposure and consumption data; the multi-dimensional data includes one or more of the following: product dimension, advertiser dimension, and product placement dimension.

3. The method according to claim 2, characterized in that, The step of determining the value of each candidate target product based on multi-dimensional exposure and consumption data of each candidate target product in the database includes: If the cumulative exposure of the product dimension of the candidate target product is greater than or equal to the effective exposure threshold of the product dimension, then the value of the candidate target product is determined based on the cumulative consumption and cumulative exposure of the product dimension of the candidate target product. If the cumulative exposure of the candidate target product in the product dimension is less than the effective exposure threshold of the product dimension, and the cumulative exposure of the candidate target product in the advertiser dimension is greater than or equal to the effective exposure threshold of the candidate target product in the advertiser dimension, then the value of the candidate target product is determined based on the cumulative consumption of the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, and the cumulative exposure of the product dimension. If the cumulative exposure of the candidate target product in the product dimension is greater than or equal to the effective exposure threshold of the product dimension, the cumulative exposure of the candidate target product in the advertiser dimension is less than the effective exposure threshold of the advertiser dimension, and the cumulative exposure of the candidate target product in the product placement dimension is greater than or equal to the effective exposure threshold of the product placement dimension, then the value of the candidate target product is determined based on the cumulative consumption of the advertiser dimension, the cumulative consumption of the product dimension, the cumulative exposure of the advertiser dimension, the cumulative exposure of the product dimension, the cumulative consumption of the product placement dimension, and the cumulative exposure of the product placement dimension.

4. The method according to claim 1, characterized in that, Based on the real-time operational status data of the bidding platform and the queue length control strategy, the control length of the original recall queue is determined, including: The real-time operating status data of the bidding platform is obtained according to a preset time window. The operating status data includes: timeout rate and anomaly rate. Based on the real-time operating status data of the bidding platform in the current time window and the preset threshold indicators, the current control method of the original recall queue is determined, and the control method includes: reducing the queue length or increasing the queue length. The current control length of the original recall queue is determined based on the current control method, the first control ratio corresponding to the control method, and the length of the recall queue after processing in the previous time window.

5. The method according to claim 4, characterized in that, The step of determining the current control length of the original recall queue based on the current control method, the control ratio corresponding to the control method, and the length of the recall queue after processing in the previous time window includes: The target length is determined based on the length of the recall queue after processing in the previous time window and the control ratio corresponding to the control method. The current control length of the original recall queue is calculated based on the current control method, the length of the recall queue after the previous time window processing, and the target length.

6. The method according to claim 1, characterized in that, The step of truncating the original recall queue according to the adjusted length and a preset truncation strategy to obtain the target recall queue includes: The truncation length of the original recall queue is determined based on the length of the original recall queue and the adjustment length. Based on the distributor to which each candidate target product belongs in the original recall queue, the candidate target products are grouped to obtain sub-queues corresponding to each distributor. The product truncation length for each distributor is determined based on the data length of the sub-queue corresponding to each distributor and the truncation length of the original recall queue. Based on the product truncation length corresponding to each advertiser, the candidate target products in the sub-queue corresponding to each advertiser are truncated to obtain the retained product information corresponding to each advertiser. The target recall queue is obtained based on the retained product information of each distributor.

7. The method according to claim 6, characterized in that, The step involves truncating each candidate target product in the sub-queue corresponding to each advertiser based on the product truncation length, to obtain the retained product information for each advertiser, including: Based on the value of each candidate target product in the sub-queue corresponding to the advertiser, the candidate target products are sorted to obtain the target sub-queue corresponding to the advertiser. The first retention quantity is determined based on the length of the sub-queue corresponding to the delivery party and the preset retention ratio; Based on the first reserved quantity, select a plurality of first target products from the target sub-queue corresponding to the delivery party according to the sorting result of each candidate target product; Based on the product cut-off length corresponding to the advertiser and the first retention quantity, a second retention quantity is determined, and based on the second retention quantity, multiple second target products of the second retention quantity are selected from the remaining candidate target products in the target sub-queue corresponding to the advertiser according to a random algorithm. The retained product information corresponding to the distributor includes a first number of multiple first target products and a second number of multiple second target products.

8. A data processing apparatus, characterized in that, include: Determine the module and the processing module; The determining module is used to determine the value of each candidate target product based on the data of each candidate target product recorded in the database; The determining module is used to determine the original recall queue based on the value of each candidate target product; the original recall queue includes multiple candidate target products, and each candidate target product is sorted according to its value. The determining module is used to determine the control length of the original recall queue based on the real-time operating status data of the bidding platform and the queue length control strategy. The processing module is used to truncate the original recall queue according to the controlled length and the preset truncation strategy to obtain the target recall queue.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the program instructions to perform the data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which is executed by a processor to perform the data processing method as described in any one of claims 1 to 7.

Images