An advertisement traffic anti-cheating method based on reinforcement learning

Abstract

This invention discloses a reinforcement learning-based method for combating advertising traffic fraud, specifically in the field of advertising fraud prevention technology. It addresses the problem that existing methods primarily rely on single-rule decisions or static model identification, making it difficult to continuously and hierarchically process suspicious but unconfirmed advertising traffic. The method aggregates advertising event chain data by observation window to form observation window traffic cluster records. Based on a risk boundary table, it identifies suspicious but unconfirmed traffic clusters whose risk values ​​reach the lower boundary of suspicion but are below the upper boundary of interception. Then, it combines the immediate feedback field, conversion feedback field, and review feedback field from the delayed feedback ledger to generate a reinforcement learning state vector. This vector, along with an action boundary table, generates an action mask and limits hierarchical processing actions. This achieves continuous identification, hierarchical processing, and boundary constraint control of suspicious but unconfirmed advertising traffic, avoiding premature interception or continuous consumption of abnormal traffic due to reliance on single-rule decisions.

Description

Technical Field

[0001] This invention relates to the field of advertising anti-fraud technology, specifically to an advertising traffic anti-fraud method based on reinforcement learning. Background Technology

[0002] As the internet advertising delivery chain continues to expand, ad traffic typically goes through multiple stages, including ad requests, impressions, clicks and redirects, landing page visits, conversion feedback, settlement and billing, and traffic quality verification. To identify issues such as fake clicks, abnormal impressions, device aggregation, proxy address access, and non-genuine conversions, existing anti-fraud systems generally collect ad event logs and generate risk assessment results based on fields such as media source, ad placement, device identifier, network address, click-through rate, visit rate, conversion rate, and cost. Common methods include setting rule thresholds, maintaining blacklists, statistically analyzing device or address aggregation, and calling risk scoring models. When an ad traffic record reaches preset blocking conditions, the system directly performs actions such as blocking, deducting traffic, limiting frequency, suspending settlement, or transferring the case to manual review.

[0003] The methods described above can effectively identify abnormal traffic with obvious characteristics. However, in actual campaigns, much traffic cannot be definitively judged as normal or fraudulent at the current moment. For example, some traffic may exhibit clustering characteristics under the same media source, the same ad placement, the same device fingerprint cluster, or the same internet address range, but its conversion feedback has not yet arrived, the verification conclusion has not been reached, and the settlement status may still be pending confirmation. In this case, if interception is directly based solely on real-time fields such as current impressions, clicks, and visits, genuine conversion traffic may be prematurely blocked; if measures are taken only after subsequent conversion feedback or verification results arrive, abnormal traffic may continue to consume advertising budget during the waiting period. Therefore, for suspicious unconfirmed traffic that falls between normal traffic and clearly fraudulent traffic, existing single-rule judgment or static scoring methods cannot simultaneously ensure both timely handling and reliable judgment.

[0004] Furthermore, the existing ad fraud prevention systems often operate in a fragmented manner, with the ad placement observation window, risk assessment, handling actions, and follow-up feedback all disconnected. While some systems can calculate risk scores within the current observation window, the handling methods after the risk score reaches a suspicious range are relatively fixed, lacking a delayed feedback ledger for continuous tracking of suspicious traffic. Although some systems can receive conversion feedback, manual review, or traffic quality review results, these follow-up feedbacks are usually only used as a basis for manual verification or settlement adjustments, failing to establish a continuous link with previous suspicious states, handling actions, and the next round of ad placement strategies. As a result, when similar media sources, ad placements, device clusters, or address clusters reappear, the system may still make repeated judgments based on the original risk boundaries and handling strategies, making it difficult to promptly correct subsequent judgment boundaries based on the handling results of the previous observation window.

[0005] Therefore, existing technologies still have the following problems: For ad traffic with risk values ​​in the suspicious range but not yet meeting the conditions for direct interception, there is a lack of a closed-loop mechanism that can integrate ad event chain data within the ad delivery observation window into stable traffic cluster records, and continuously associate the suspicious status, handling actions, and feedback results after the real-time feedback, conversion feedback, and review feedback have gradually matured, and then use them for subsequent risk boundary and handling strategy correction. This leads to the problem that suspicious and unconfirmed ad traffic is prone to premature interception, delayed handling, or repeated misjudgment. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a reinforcement learning-based method for combating advertising traffic fraud, thereby resolving the problems mentioned in the background section.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a method for combating advertising traffic fraud based on reinforcement learning, comprising: S1. Collect advertising event chain data according to the placement observation window, and aggregate it into observation window traffic cluster records based on media source, ad position, device fingerprint cluster, and Internet address range; S2. Match the traffic cluster records in the observation window to the risk boundary table, and lock the suspicious unconfirmed traffic clusters whose risk values ​​reach the suspicious lower boundary and are lower than the interception upper boundary. S3. Establish a delayed feedback ledger for suspicious and unconfirmed traffic clusters, write the instant feedback field, conversion feedback field, and review feedback field, and form feedback maturity mark and feedback credibility mark; S4. Generate reinforcement learning state vectors based on suspicious state records and delayed feedback ledgers, and generate action masks based on action boundary tables to limit graded handling actions. S5. Submit the reinforcement learning state vector and action mask to the policy model to generate a disposal action record, and write the action effective time, action deduplication key, and action deadline into the disposal action record. S6. When both the feedback maturity mark and the feedback credibility mark meet the write-back conditions, generate a reward value, update the strategy model, and write back the risk boundary, action cooldown conditions, and strategy version to the next deployment observation window.

[0008] Furthermore, S1 includes: The server receiving time is used as the event window time for advertising event chain data; Time is assigned to windows according to the following rules: the starting point of the window is included in the current delivery observation window, the ending point of the window is not included in the current delivery observation window, and events falling at the ending point of the window are assigned to the next delivery observation window. The criteria for identifying duplicates are formed by event number, advertising campaign identifier, media source, advertising space, device fingerprint cluster, Internet address range, and event window time. Events with consistent media sources, ad placements, device fingerprint clusters, internet address ranges, and fields are aggregated into observation window traffic cluster records in the order of time windowing, source matching, ad placement matching, device fingerprint clusters, internet address ranges, field denoising, and field completion.

[0009] Furthermore, S2 includes: Read the submitted traffic cluster records in the observation window that have been aggregated and marked as complete; Confirm the applicable start and end times of the risk boundary table based on the observation window number; Risk boundary tables are matched based on advertising campaign identifiers, media sources, ad placements, device fingerprint cluster formation sources, and internet address range sources; Risk values ​​are generated based on a fixed field order and a fixed scale for forming the risk value definition; When the risk value reaches the lower boundary of suspicion but is below the upper boundary of interception, the corresponding suspicious unconfirmed traffic cluster record will be written into the suspicious record area.

[0010] Furthermore, S3 includes: Create a delayed feedback ledger in the feedback ledger area by using the observation window number, the locked batch number, and the deduplication key for the locked record; According to the feedback rules, the feedback deadline, the review waiting boundary, and the list of authorized sources are locked. Add immediate feedback, conversion feedback, and review feedback fields to the delayed feedback ledger; Based on field gap markers, converted records awaiting verification, feedback deadlines, review waiting boundaries, and review conflict logs, feedback maturity markers are generated. Based on the list of authorized sources, the complete status of associated keys, and the verification of conflict traces, a reliable feedback marker is generated.

[0011] Furthermore, S4 includes: Deduplicat the locked record and associate it with the suspicious status record and the delayed feedback ledger; Read the risk value, lower boundary of suspicion, upper boundary of interception, lock time, media source, ad slot, device fingerprint cluster and Internet address range from the suspicious status record; Read the immediate feedback field, conversion feedback field, review feedback field, feedback maturity flag, and feedback credibility flag from the delayed feedback ledger; A fixed-length reinforcement learning state vector is formed by following the fixed field order of the state vector rule version.

[0012] Furthermore, S4 also includes: Read the effective action boundary table and match the action boundary table according to media source, ad position, feedback maturity flag, feedback credibility flag, risk value range and lock duration range; Generate an action mask based on a fixed sequence of graded handling actions; Exclude actions corresponding to prohibited tags based on action masks, and retain actions corresponding to allowed tags based on action masks; The reinforcement learning state vector, action mask, and the defined hierarchical handling actions are written into the state-action record area.

[0013] Furthermore, S5 includes: Read the submitted reinforcement learning state vector, action mask, and limited hierarchical processing actions from the state-action record area; The candidate order for each level of action is formed by reading the state vectors fixed in the strategy model version and the action candidate scoring scale; Eliminate tiered actions with prohibited markers in the action mask; Generate a record of actions that includes the first-ranked action in the remaining candidate sorting, and write the action's effective time, deduplication key, and deadline.

[0014] Furthermore, S6 includes: Read the action record and delay feedback ledger after the action deadline has passed; The write-back rules are used to lock the write-back conditions, the definition of the reward value field, the positive update threshold, the negative update threshold, and the policy version release definition. A reward value is generated when both the feedback maturity mark and the feedback trust mark meet the write-back conditions. The reinforcement learning state vector reference, action mask reference, generated hierarchical disposal action, reward value, policy model version, and write-back rule version are written into the policy update record area, and a new policy version is formed by appending.

[0015] Furthermore, S6 also includes: Based on the reward value, risk value, suspicious lower boundary, and interception upper boundary, a risk boundary version is formed according to the boundary step table in the write-back rule version; Based on the generated graded handling actions, action effective time, action deadline, reward value, and review feedback fields, an action cooldown condition version is formed; Write the new strategy version, risk boundary version, and action cooldown condition version to the write-back record area of ​​the next deployment observation window.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. By aggregating advertising event chain data into observation window traffic cluster records, traffic clusters whose risk values ​​reach the suspicious lower boundary but are below the interception upper boundary are locked as suspicious unconfirmed traffic clusters. The delayed feedback ledger includes immediate feedback, conversion feedback, and review feedback fields. Furthermore, reinforcement learning state vectors are generated based on the suspicious status records and delayed feedback ledger, and action masks are generated based on the action boundary table to limit the hierarchical handling actions. This achieves the goal of continuous, hierarchical, and boundary-constrained handling of suspicious but unconfirmed advertising traffic, avoiding premature interception or allowing abnormal traffic to continue to be consumed by relying solely on one-time rule judgments.

[0017] 2. By generating reward values ​​when both the feedback maturity marker and the feedback credibility marker meet the write-back conditions, updating the strategy model, and writing back the risk boundary, action cooldown conditions, and strategy version to the next delivery observation window, the goal of converting conversion feedback, review feedback, and handling results into strategy boundaries that can be called by the next delivery observation window is achieved. This enables the anti-fraud process of advertising traffic to be corrected in a closed loop with delayed feedback, reducing repeated misjudgments and repeated handling under the same source, the same ad position, the same device fingerprint cluster, and the same Internet address range. Attached Figure Description

[0018] Figure 1 A diagram illustrating the data sources for the advertising event chain and the collection of data through the placement observation window; Figure 2 A diagram illustrating the logic for aggregation and accounting of traffic cluster records within the observation window; Figure 3 A schematic diagram illustrating risk boundary table matching and locking of suspicious unconfirmed traffic clusters; Figure 4 A schematic diagram illustrating the establishment of the delayed feedback ledger and the formation of feedback markers; Figure 5 A schematic diagram illustrating the formation of state vectors for reinforcement learning; Figure 6 A schematic diagram illustrating the generation of action masks and the hierarchical handling of action limitations. Figure 7 A schematic diagram is generated to record the strategy model invocation and handling actions; Figure 8 This is a diagram illustrating reward value generation and strategy model updates. Figure 9 This is a diagram illustrating the write-back of risk boundaries, action cooldown conditions, and strategy versions to the next deployment observation window. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example: Combined with Appendix Figures 1-9 This embodiment provides a reinforcement learning-based method for combating advertising traffic fraud, including: S1. Collect advertising event chain data according to the placement observation window, and aggregate it into observation window traffic cluster records based on media source, ad placement, device fingerprint cluster, and internet address range. The specific implementation is as follows: The advertising anti-fraud system receives advertising event chain data according to the placement observation window. The placement observation window is a continuous collection period defined during the effective period of the same advertising campaign. The start point of the window is included in the current placement observation window, while the end point of the window is not included in the current placement observation window. Events that fall exactly at the end point of the window are included in the next placement observation window. The advertising event chain data comes from the exposure logs of the media traffic access layer, the request logs of the advertising request service, the click logs of the click redirect service, the access logs of the landing page tracking service, the conversion logs of the conversion feedback service, the consumption logs of the settlement side, and the traffic quality feedback logs. The collection interval is from the start point of the current placement observation window to the cutoff confirmation boundary after the end point of the current placement observation window. Events arriving after the cutoff confirmation boundary do not rewrite the already submitted observation window traffic cluster records, but only generate append correction records and retain the original event number, arrival time, and the placement observation window number to which they belong.

[0021] The event occurrence time in the advertising event chain data is based on the server's receiving time. The client's reported time is only corrected when the server's receiving time is empty and the log source has a source version record. The corrected event occurrence time is still written into the server's receiving time correction mark. When the same event appears repeatedly in multiple log sources, the event number, advertising campaign identifier, media source, ad position, device fingerprint cluster, Internet address range, and event occurrence time are used to form a duplicate identification criterion. Events that have already been recorded are not counted again, and later events enter the duplicate reporting record area.

[0022] The media source is taken from the media source number, application package name, site domain name and channel mark, and is merged into a unified source number through the media access configuration table; when the media source number, application package name, or site domain name is missing and the corresponding relationship cannot be found in the media access configuration table, the event is not merged into the existing media source, but transferred to the source record to be supplemented.

[0023] The ad placement is determined by the ad placement number, page position, display style, and ad placement version. The ad placement version is based on the media access configuration table record that is currently active in the current campaign observation window. If the ad placement version is switched within the same campaign observation window, the events before and after the switch will form separate traffic cluster records in the observation window, and will not be merged across ad placement versions.

[0024] Device fingerprint clusters are primarily formed from de-identified encrypted device identifiers. The same encrypted device identifier corresponds to the same device fingerprint cluster within the same delivery observation window. When an encrypted device identifier is missing, the merging condition is that the same session encryption identifier, system type, browser tag, screen resolution segment, and network type are completely consistent within the same delivery observation window. If any of these fields are missing, no device fingerprint cluster is formed, and the device fingerprint cluster is not derived by reverse engineering the Internet address segment. Device fingerprint clusters only store the cluster number and the source mark of their formation, and do not store plaintext device identifiers, telephone numbers, ID numbers, or natural person identity fields.

[0025] Internet address ranges are formed by organizing the requested Internet address through the address range rule table and recorded at the address range level. The complete requested Internet address is not stored. When the same requested Internet address hits the proxy identification database, the Internet address range is also written with a proxy source tag. Events that are not hit in the address range rule table are entered into the address range record to be supplemented. The Internet address range is not replaced by the region name.

[0026] The processing order is fixed as follows: time-based windowing, source matching, ad placement matching, device fingerprint cluster formation, internet address range formation, field denoising, field completion, and recording of traffic clusters in the observation window. Field denoising only removes records marked with test traffic, internal monitoring, duplicate reporting, broken event numbers, or out-of-bounds event occurrence times; these removed records are not included in the count statistics. Field completion only allows the media source and ad placement version to be completed from the media access configuration table, and only allows the internet address range to be completed from the address range rule table. It does not automatically complete event occurrence time, ad campaign identifier, device fingerprint cluster, number of impressions, number of clicks, number of landing page visits, number of conversion feedbacks, and cost.

[0027] The observation window traffic cluster records are aggregated according to media source, ad placement, device fingerprint cluster, and internet address range. Events with completely identical aggregation fields in the same observation window are counted in the same observation window traffic cluster record. If any aggregation field is different, different observation window traffic cluster records are formed. The observation window traffic cluster records are appended to the observation window record area of ​​the anti-fraud feature database. The record content includes observation window number, media source, ad placement, device fingerprint cluster, internet address range, number of impressions, number of clicks, number of landing page visits, number of conversion feedbacks, cost, event source version, aggregation completion time, record version, and record deduplication key. The unit of number of impressions is times, the unit of cost is the currency of the advertising account settlement, the record version consists of observation window number, aggregation batch number, and configuration table version number, and the record deduplication key is generated by observation window number, media source, ad placement, device fingerprint cluster, and internet address range.

[0028] When submitting the current step, submit in ascending order of observation window number. Only one submitted observation window traffic cluster record can exist for the same observation window number and the same deduplication key. Changes caused by late log entries, configuration updates, and address range updates must not overwrite submitted records. Only append correction records can be generated and point to the original record version.

[0029] When reading downstream processes, the data is based on the completion mark, record version, and location of the traffic cluster record in the observation window. Source records to be supplemented, address range records to be supplemented, records of equipment fingerprint clusters that have not been formed, and duplicate reports are not included in the formal observation window traffic cluster record.

[0030] Security compliance boundaries are limited to not writing plaintext device identifiers, complete request Internet addresses, natural person identity fields, and unauthorized media side fields; the media access configuration table, address range rule table, event source version, and aggregation batch are all written with version numbers and timestamps to form source traces of traffic cluster records in the current delivery observation window.

[0031] Preferably, the observation window can be set to 15 minutes, the confirmation deadline can be set to 3 minutes after the end of the observation window, the server receiving time correction tolerance can be set to 2 seconds, the single batch aggregation delay limit can be set to 30 seconds, the number of retries for the same log source can be set to 3, and the retry interval can be set to 10 seconds; for example, during the observation window of an advertising campaign from 09:00:00 to 09:15:00, 120,000 impressions, 6,400 clicks, 5,100 landing page visits, and 380 conversion feedbacks are received from ad slot B of media source A, and then processed through device fingerprint cluster C and the network. After the internet address range is aggregated, a single observation window traffic cluster record is formed. If the ad slot version is switched at 09:08:00, a single observation window traffic cluster record is formed from 09:00:00 to 09:08:00, and another single observation window traffic cluster record is formed from 09:08:00 to 09:15:00. Alternatively, when a stable encrypted device identifier cannot be obtained on the webpage, a device fingerprint cluster can be formed using the same session encryption identifier, system type, browser tag, screen resolution range, and network type. However, the complete request internet address must not be used to replace the device fingerprint cluster.

[0032] S2. Match the traffic cluster records in the observation window to the risk boundary table, and lock in suspicious unconfirmed traffic clusters whose risk values ​​reach the suspicious lower boundary but are below the interception upper boundary. The specific implementation is as follows: The anti-fraud judgment process reads the observation window traffic cluster records that have been submitted in the previous stage and have the aggregation completion mark, and matches them one by one with the risk boundary table within the submission batch of the same observation window. The observation window traffic cluster records come from the observation window record area of ​​the anti-fraud feature library. When reading, the observation window number, record version, and record deduplication key are used as the criteria. Records without the aggregation completion mark, records to be supplemented, and duplicate reports are not included in this step to avoid records that have not yet formed a stable aggregation standard from entering the risk judgment.

[0033] The risk boundary table records the judgment rules that are currently in effect within the current campaign observation window. The table must include at least the table version, applicable start time, applicable end time, advertising campaign identifier, media source, ad placement, device fingerprint cluster formation source, internet address range source, risk value formation criteria, suspicious lower boundary, interception upper boundary, and handling criteria for missing fields. The risk value is an integer record formed according to a fixed scale in the risk boundary table, with a value range of 0 to 1000. The larger the value, the closer the traffic cluster record in the observation window is to the interception range under the current version of the risk boundary table.

[0034] The risk boundary table is compiled and released based on historical manual review results, traffic quality feedback results, advertising campaign placement baselines, and media access configurations confirmed by the operations side. This step does not rewrite the risk boundary table; it only matches the already effective version of the table.

[0035] During matching, the applicable start and end times of the risk boundary table are first confirmed using the observation window number. If the time period corresponding to the observation window number does not fall completely within the applicable time of the table version, the table version is not used. Then, the advertising campaign identifier, media source, ad placement, device fingerprint cluster formation source, and Internet address range source are compared in sequence. If all fields are completely consistent, the risk boundary table record is used.

[0036] If a traffic cluster record in the same observation window hits multiple risk boundary table records simultaneously, the record that hits all four criteria—ad campaign identifier, media source, ad placement, device fingerprint cluster formation source, and internet address range—will be prioritized. If multiple records still exist, the record with the latest publication time and the largest table version number will be used. If the media source is hit but the ad placement is not, the general boundary record for the ad placement under the same media source that has been version locked will be used. If the media source is not hit, the traffic cluster record in the observation window will not be included in the risk value comparison and will be registered as a boundary record pending verification.

[0037] The risk value is formed after the traffic cluster records in the observation window are matched with the risk boundary table. When forming the risk value, only the number of impressions, clicks, landing page visits, conversion feedbacks, consumption amount, device fingerprint clusters, Internet address ranges, and the risk values ​​in the matched risk boundary table that have been recorded in the traffic cluster records of the observation window are used. The number of impressions, clicks, landing page visits, and conversion feedbacks are all measured in units of "time". The consumption amount is settled in the currency of the advertising account. Device fingerprint clusters and Internet address ranges are only used as aggregation criteria and do not infer the identity of natural persons.

[0038] The risk value is recorded in the risk boundary table with a fixed field order and fixed scale. The fixed field order is as follows: the degree of concentration of clicks relative to exposure, the degree of landing page visits relative to clicks, the degree of deviation of conversion feedback relative to clicks, the intensity of spending relative to exposure, the degree of concentration of the same device fingerprint cluster in the same media source ad slot, and the degree of concentration of the same internet address range in the same media source ad slot. Each item has a low-risk scale, a medium-risk scale, a high-risk scale, and a corresponding integer score pre-set in the risk boundary table. The starting boundary of each scale is included in the current scale, while the ending boundary is not included. The ending boundary of the highest scale is included in the highest scale. The same field value can only fall into one scale. When the conversion and return count is 0, the valid value 0 is used for scale matching and is not considered as a missing field. The anti-fraud judgment process reads the observation window traffic cluster records item by item according to the above fixed field order. After each item falls into the corresponding scale, the corresponding integer score is obtained. Then, the risk value is summarized according to the field order locked in the risk boundary table. If the summarized risk value exceeds 1000, it is recorded as 1000.

[0039] When the number of exposures is 0, the number of clicks is missing, the number of landing page visits is missing, the number of conversion feedbacks is missing, the amount spent is missing, the device fingerprint cluster is not formed, the Internet address range is not formed, or the record version is inconsistent with the version applicable to the risk boundary table, the risk value will be stopped from being generated, and the traffic cluster record in that observation window will be registered as a record to be verified.

[0040] The suspected lower boundary is the lower limit of the allowed entry into the suspected observation range in the risk boundary table, and the interception upper boundary is the upper limit of the allowed entry into the interception range in the risk boundary table. Both are locked with the table version and are recorded using integers. Within the same risk boundary table record, the suspected lower boundary must be less than the interception upper boundary. If the suspected lower boundary is greater than the interception upper boundary, or the suspected lower boundary is equal to the interception upper boundary, then the table version cannot be used for this step, and the relevant observation window traffic cluster record is registered as a boundary pending verification record.

[0041] When locking, the observation window traffic cluster record with formed risk value is compared with the suspected lower boundary and interception upper boundary in the corresponding risk boundary table; when the risk value reaches the suspected lower boundary and is lower than the interception upper boundary, it is locked as a suspected unconfirmed traffic cluster; when the risk value is lower than the suspected lower boundary, it is not locked as a suspected unconfirmed traffic cluster; when the risk value reaches the interception upper boundary, it is not included in the suspected unconfirmed traffic cluster record formed in this step.

[0042] The suspicious unconfirmed traffic cluster records generated in this step are appended to the suspicious record area of ​​the anti-fraud feature database. The record content includes the observation window number, the deduplication key of the original observation window traffic cluster record, media source, ad slot, device fingerprint cluster, Internet address range, risk value, suspicious lower boundary, interception upper boundary, risk boundary table version, locking time, locking batch number, and locking record deduplication key. The locking record deduplication key is jointly generated by the observation window number, the original observation window traffic cluster record deduplication key, and the risk boundary table version. Only one submitted suspicious unconfirmed traffic cluster record can exist with the same locking record deduplication key. Duplicate locking results do not overwrite the original record, but only write duplicate locking traces.

[0043] Once a suspicious unconfirmed traffic cluster record is submitted, it can be read by subsequent steps according to the observation window number, locked batch number, and locked record deduplication key. If the risk boundary table is temporarily unreadable, the traffic cluster records in the observation window that have been read will remain in an undetermined state and the reason for the table reading failure will be recorded, without interrupting the locking of other matched records in the same batch.

[0044] During on-site verification, locked suspicious unconfirmed traffic cluster records can be extracted and their original observation window traffic cluster records, risk boundary table version, risk value formation criteria, suspicious lower boundary, interception upper boundary, and comparison results can be checked. Only records that confirm that the source of the risk value is traceable, the boundary version is not mixed, and the risk value reaches the suspicious lower boundary but is lower than the interception upper boundary are included in the suspicious unconfirmed traffic cluster records.

[0045] Preferably, the risk value can be set to an integer scale from 0 to 1000, the lower boundary of suspicion can be set to 420, the upper boundary of interception can be set to 760, the single batch lockout delay limit can be set to 20 seconds, the number of retries for reading the risk boundary table can be set to 3, and the retry interval can be set to 5 seconds; for example, a traffic cluster in a certain observation window records 120,000 impressions, 6,400 clicks, 5,100 landing page visits, 380 conversion feedbacks and corresponding consumption amount in the observation window from 09:00:00 to 09:15:00. After matching the risk boundary table versions corresponding to media source A, ad placement B, device fingerprint cluster formation source C, and internet address range source D, according to the... Within the table version, fixed fields are used to obtain the clustering score of clicks relative to exposure, the landing page visit relative to click engagement score, the conversion feedback relative to click deviation score, the consumption intensity score of consumption amount relative to exposure, the concentration score of fingerprint clusters on the same device, and the concentration score of the same internet address range. After summarizing, a risk value of 610 is formed. This risk value reaches 420 but is lower than 760, so it is locked as a suspicious unconfirmed traffic cluster. Alternatively, when the sample size of the ad placement dimension is insufficient to publish a risk boundary table record separately, the general boundary record of the ad placement locked under the same media source can be used for matching, but risk boundary table records should not be called across media sources.

[0046] S3. Establish a delayed feedback ledger for suspicious unconfirmed traffic clusters, writing the immediate feedback field, conversion feedback field, and review feedback field to form feedback maturity markers and feedback credibility markers. The specific implementation is as follows: The anti-fraud ledger process reads submitted suspicious unconfirmed traffic cluster records in the suspicious record area and establishes a delayed feedback ledger for each suspicious unconfirmed traffic cluster according to the observation window number, locked batch number, and locked record deduplication key. The delayed feedback ledger is an append-only record that continuously registers the arrival of feedback around the same suspicious unconfirmed traffic cluster. It is established in the feedback ledger area of ​​the anti-fraud feature library. Its effective period starts from the locking time of the suspicious unconfirmed traffic cluster and extends to the corresponding feedback deadline and review waiting boundary. The feedback deadline, review waiting boundary, authorized source list, complete field definition, and feedback conflict definition are all locked according to the feedback rule version.

[0047] When the ledger is created, suspicious unconfirmed traffic cluster records, exposure logs, click logs, landing page access logs, conversion feedback logs, traffic quality feedback records, settlement side status records, and review records are read. Suspicious unconfirmed traffic cluster records without deduplication keys for locked records are not included in the delayed feedback ledger. Delayed feedback ledgers with deduplication keys for the same locked record are not created again; only the batch number of this feedback arrival is appended.

[0048] After the delayed feedback ledger is established, the immediate feedback field is written first. The immediate feedback field records the number of impressions, clicks, landing page visits, cost, initial traffic quality feedback status, and current status of the suspicious unconfirmed traffic cluster that has been recorded before the lock time and whose event time belongs to the original ad observation window. The number of impressions, clicks, and landing page visits are all in units of times. The cost uses the advertising account settlement currency. The initial traffic quality feedback status is obtained by associating with the traffic quality feedback record according to the same observation window number, media source, ad position, device fingerprint cluster, and Internet address range. The current status of the settlement side is obtained by associating with the settlement side status record according to the advertising campaign identifier and the deduplication key of the original observation window traffic cluster record.

[0049] If any of the following fields are missing from the instant feedback field: number of impressions, number of clicks, number of landing page visits, or amount spent, the delayed feedback ledger will still be created, but a field gap marker will be written. Events that are later than the original campaign observation window and belong to the same suspicious unconfirmed traffic cluster will not be written to the instant feedback field, but will be entered into the conversion feedback field or the review feedback field according to their source.

[0050] The conversion feedback field is appended when the conversion feedback log arrives, recording the number of conversion feedbacks received before the feedback deadline, the conversion feedback time, the conversion type, the source of conversion confirmation, and the conversion withdrawal status. If no conversion feedback log is received by the feedback deadline and there are no conversion records pending verification, the conversion feedback field is set to 0 for the number of conversion feedbacks received, and a "no conversion reached" flag is added. The "no conversion reached" flag is not the same as a field gap flag. The conversion feedback field is associated with conversion feedback logs according to the advertising campaign identifier, media source, ad placement, device fingerprint cluster, internet address range, and click event number. If the click event number is missing, the conversion attribution is not inferred, and the conversion is transferred to the conversion pending verification record. Conversion feedback logs arriving after the feedback deadline must not overwrite the already written conversion feedback field; only late conversion traces are generated.

[0051] The review feedback field is added when manual review records, media complaint review records, and traffic quality review records arrive, recording the review source, review conclusion, review time, review batch number, and review record version. The review feedback field is associated with review records based on the deduplication key of the locked record and the deduplication key of the original observation window traffic cluster record. If the review source is not listed in the authorized source list, it is not written to the review feedback field, and only a review source pending verification record is generated. If a review record in the authorized source list is missing any of the deduplication keys of the locked record, the original observation window traffic cluster record, or the review batch number, it is not written to the review feedback field, and a review associated pending verification record is generated. When the same review batch number arrives repeatedly, the content written the first time is retained, and subsequent records are written to the duplicate review record. When different authorized sources give similar review conclusions for the same suspicious unconfirmed traffic cluster, one conclusion is not selected to overwrite the other conclusion, and a review conflict record is written.

[0052] The feedback maturity flag is formed after the current batch is appended to the immediate feedback field, conversion feedback field, and review feedback field. The flag value is fixed as immature, partially mature, and mature. An immature flag is formed when any of the following exists: field gap flag, conversion pending verification record, review source pending verification record, review association pending verification record, or review conflict trace. A partially mature flag is formed when the immediate feedback field is complete and the current time has not yet reached the feedback deadline. A mature flag is formed when the immediate feedback field is complete, the current time has reached the feedback deadline, there are no conversion pending verification records within the feedback deadline, the current time has reached the review waiting boundary, or there is an authorized review conclusion in the review feedback field or a no-review-arrival flag is written. The no-review-arrival flag is written by the ledger process after the review waiting boundary is reached, when it confirms that the authorized source list has not returned the review record corresponding to the deduplication key of the locked record.

[0053] The feedback credibility marker is formed after the feedback maturity marker is formed, and the marker value is fixed as low credibility, medium credibility, and high credibility. A low credibility marker is formed when any of the following exists: unauthorized source, missing association key, review conflict trace, or conflict between feedback content from the same source and batch. A medium credibility marker is formed when the immediate feedback field and the conversion feedback field come from an authorized source with a version number, the association key is complete, and there is a late conversion trace but the content written within the feedback deadline is not overwritten. A high credibility marker is formed when the immediate feedback field, conversion feedback field, and review feedback field all come from an authorized source with a version number, the association key is complete, and there are no late conversion traces, duplicate review traces, or review conflict traces. A medium credibility marker is formed when only a no-review-arrival marker is written after the review waiting boundary is reached, without any authorized review conclusion.

[0054] The delayed feedback ledger is saved in an append-only manner. The ledger content includes the observation window number, the deduplication key for locked records, the deduplication key for the original observation window traffic cluster records, the immediate feedback field, the conversion feedback field, the review feedback field, the feedback maturity flag, the feedback trust flag, the feedback rule version, the ledger version, and the ledger update time. The old ledger version must not be overwritten. After the fields are appended, a new ledger version is generated and the previous ledger version number is retained.

[0055] Downstream processes read the delayed feedback ledger based on the deduplication key of the locked record, the ledger version, the feedback maturity flag, and the feedback trust flag. When the feedback rule version is temporarily unreadable, the delayed feedback ledger maintains the established account status and writes a rule read failure trace, without interrupting the establishment of accounts for other suspicious and unconfirmed traffic clusters.

[0056] The security compliance boundary is limited to not writing plaintext device identifiers, complete Internet addresses, natural person identity fields, and unauthorized review materials. The feedback field only saves the number of times, amount, status, version, and time associated with suspicious and unconfirmed traffic clusters.

[0057] Preferably, the feedback deadline can be set to 24 hours after the lock time, the review waiting boundary can be set to 48 hours after the lock time, the ledger append delay limit can be set to 30 seconds, the number of retries for the same feedback source can be set to 3, and the retry interval can be set to 10 seconds. For example, if a suspicious unconfirmed traffic cluster is locked at 09:16:00, the delayed feedback ledger first writes 120,000 impressions, 6,400 clicks, 5,100 landing page visits and corresponding consumption amounts. Within 24 hours, 380 conversion feedbacks are received, and within 48 hours, authorized review records are received. If there are no field gap markers, conversion records pending verification, review source records pending verification, review association records pending verification, review conflict traces, late conversion traces, or duplicate review traces, then a mature marker and a high-credibility marker are formed. Alternatively, in media sources without manual review records, the traffic quality review record can serve as the source for the review feedback fields, but this source must be pre-written into the authorized source list of the feedback rule version.

[0058] S4. Generate reinforcement learning state vectors based on suspicious state records and delayed feedback ledgers, and generate action masks based on action boundary tables to limit graded handling actions. The specific implementation is as follows: The state organization process reads suspicious state records and delayed feedback ledgers, and associates the two according to the lock record deduplication key to generate reinforcement learning state vectors. Suspicious state records are the suspicious unconfirmed traffic cluster records written into the suspicious record area in the second step. The state side read records in this step are read in this step. The two correspond one-to-one through the same lock record deduplication key, without changing the risk value, suspicious lower boundary, interception upper boundary and lock time formed in the second step.

[0059] Suspicious status records come from the suspicious record area of ​​the anti-fraud feature database, and delayed feedback ledgers come from the feedback ledger area. When reading, the observation window number, lock batch number, lock record deduplication key, and ledger version are used as the criteria. Delayed feedback ledgers that have not formed feedback maturity or feedback credibility marks are not included in this step. When the same lock record deduplication key corresponds to multiple ledger versions, the ledger version with the latest update time and that has not been revoked is read, and the reading criteria are written into the status sorting batch number.

[0060] The suspicious status log provides risk value, suspicious lower boundary, interception upper boundary, media source, ad placement, device fingerprint cluster, internet address range, lock time, and risk boundary table version; the delayed feedback ledger provides immediate feedback field, conversion feedback field, review feedback field, feedback maturity flag, feedback trust flag, feedback rule version, and ledger update time.

[0061] The reinforcement learning state vector is a fixed-length numerical record formed according to the state vector rule version. Under the same state vector rule version, the number of fields, the field positions, and the numbering table are fixed and will not be temporarily added or removed with a single record. The field order should at least include the risk value position, boundary interval position, immediate feedback complete position, conversion feedback arrival position, review feedback position, feedback maturity position, feedback credibility position, lock duration position, media source position, advertising position, device fingerprint cluster position, and internet address range position.

[0062] The risk value location is directly written as an integer risk value in the suspicious status record; the boundary interval location is converted into an interval number according to the boundary interval table in the state vector rule version, where the risk value is located between the suspicious lower boundary and the interception upper boundary; the complete location of the immediate feedback is converted into a complete number or a gap number according to whether there is a field gap mark in the immediate feedback field; the arrival location of the conversion feedback is converted into a corresponding number according to the number of conversion feedback returns and late conversion traces in the conversion feedback field; the review feedback location is converted into a corresponding number according to the authorized review conclusion, no review arrival mark, and review conflict trace; the feedback maturity location is converted into an immature number, a partially mature number, and a mature number according to the feedback maturity mark; the trusted feedback location is converted into a low-trust number, a medium-trust number, and a high-trust number according to the feedback trust mark; the lock duration location is the lock duration interval number in the state vector rule version according to the interval between the status sorting time and the lock time; the media source, advertising space, device fingerprint cluster, and Internet address range are converted into a fixed number according to the number table in the state vector rule version.

[0063] The state organization process does not recalculate risk values, rewrite the delayed feedback ledger, or supplement feedback maturity and trustworthiness markers. When a suspicious state record lacks a risk value, suspicious lower boundary, interception upper boundary, or lock time, or when the delayed feedback ledger lacks a feedback maturity marker, feedback trustworthiness marker, or ledger version, or when the media source, ad slot, device fingerprint cluster, or internet address range does not match the number table in the state vector rule version, the generation of reinforcement learning state vectors is stopped, and the corresponding lock record is deduplicated and registered as a state pending verification record.

[0064] The action boundary table records action restrictions that have taken effect during the status adjustment period and are associated with the original observation window number. The table contains the action boundary table version, applicable start time, applicable end time, media source, advertising position, feedback maturity flag condition, feedback credibility flag condition, risk value range, lock duration range, allowed action flag, and prohibited action flag. The status adjustment process does not adjust the action boundary table, but only reads the action boundary table version that has already taken effect.

[0065] When matching the action boundary table, first confirm that the status processing time falls within the applicable start time to applicable end time of the action boundary table, and then compare the media source, ad placement, feedback maturity flag, feedback credibility flag, risk value range, and lock duration range in sequence. If all are matched, the corresponding action boundary table record is used. If the ad placement is not matched but the media source is matched, the general action boundary record of the ad placement under the same media source that has been version locked is used. If the media source is not matched, no action mask is generated and it is registered as an action boundary record to be verified.

[0066] The action mask is an action permission record formed according to the fixed sequence of graded handling actions. The fixed sequence of graded handling actions is: release observation, frequency limit, reduce bidding weight, trigger secondary verification, postpone settlement, deduction mark, interception, and escalation review. Each position in the action mask only records an allow mark or a prohibit mark. When an allow action mark is hit, an allow mark is recorded; when a prohibit mark is hit, a prohibit mark is recorded; when no allow action mark is hit, a prohibit mark is recorded. When the same graded handling action hits both allow action marks and prohibit action marks, the prohibit mark shall prevail and be recorded in the action boundary conflict trace.

[0067] When limiting the tiered handling actions, tiered handling actions with prohibited markers in the action mask are excluded, and only tiered handling actions with allowed markers are retained. If no tiered handling actions are retained after exclusion, escalation review is retained as the backup tiered handling action. If escalation review is also prohibited in the action boundary table, no tiered handling action is formed after limitation, and the corresponding locked record is deduplicated and registered as an action boundary pending verification record.

[0068] The reinforcement learning state vector, action mask, and the limited graded action are appended to the state action record area of ​​the anti-cheating feature library. The record content includes the observation window number, the lock record deduplication key, the ledger version, the state vector rule version, the action boundary table version, the reinforcement learning state vector, the action mask, the limited graded action, the state processing batch number, and the processing completion time. Only one submitted record can exist for the same lock record deduplication key, the same ledger version, the same state vector rule version, and the same action boundary table version. Duplicate processing results do not overwrite the original record, but only leave a trace of duplicate processing.

[0069] The security compliance boundary is limited to the fact that the reinforcement learning state vector does not contain plaintext device identifiers, complete Internet addresses, natural person identity fields, or unauthorized review materials. Media sources, advertising slots, device fingerprint clusters, and Internet address ranges are only entered into the reinforcement learning state vector in the form of numbers.

[0070] Preferably, the upper limit of the state consolidation delay can be set to 10 seconds, the number of retries for reading the action boundary table can be set to 3, and the retry interval can be set to 5 seconds. For example, the risk value of a suspicious unconfirmed traffic cluster is 610, the lower boundary of suspicion is 420, the upper boundary of interception is 760, the feedback maturity is marked as mature, the feedback credibility is marked as highly credible, the lock duration is 25 hours, the state vector rule version converts it into a fixed-length reinforcement learning state vector, and the action mask formed after matching the action boundary table allows for observation, frequency limiting, reduction of bidding weight, triggering of secondary verification, postponement of settlement, and upgrade review, and prohibits deduction marking and interception. Alternatively, when the ad slot samples are insufficient to publish an independent action boundary table, the general action boundary record of the ad slot locked under the same media source can be used, but the action boundary table cannot be called across media sources.

[0071] S5. Submit the reinforcement learning state vector and action mask to the policy model to generate a disposal action record. Write the action effective time, action deduplication key, and action deadline into the disposal action record. The specific implementation is as follows: The strategy invocation process reads the submitted reinforcement learning state vectors, action masks, and limited hierarchical handling actions from the state action record area, and submits the reinforcement learning state vectors and action masks to the strategy model to generate handling action records. The strategy invocation process is located on the online decision-making side of the advertising anti-fraud system. The reading location is the state action record area of ​​the anti-fraud feature library. The effective period is from the completion of state sorting to the start of the executable period of the corresponding hierarchical handling action for the suspicious unconfirmed traffic cluster. When reading, the observation window number, lock record deduplication key, ledger version, state vector rule version, and action boundary table version are used as the criteria. Records that have not formed reinforcement learning state vectors, have not formed action masks, or have empty limited hierarchical handling actions are not submitted to the strategy model.

[0072] The policy model is a released reinforcement learning decision model with a model version. The model version includes a compatible state vector rule version, action boundary table version, fixed order of graded actions, and state interval numbers. The policy model version also includes a fixed state vector reading order, state position value table, action candidate scoring scale, and candidate ranking criteria. After reading each state position number according to the state vector rule version, the policy invocation process obtains the candidate scores for each graded action from the corresponding action candidate scoring scale in the policy model version, and ranks the candidates from highest to lowest score. If the candidate scores are the same, the order is determined by the fixed order of graded actions. If the state vector rule version, action boundary table version, and policy model version are incompatible, no action record is generated, and the corresponding locked record is deduplicated and registered as a policy version record to be verified.

[0073] In this step, the strategy model only generates tiered action based on the currently effective model version, without adjusting model parameters. When reading reinforcement learning state vectors, the fixed field order of the state vector rule version is followed, without adding or deleting fields, changing field positions, or replacing state records to be verified with default fields. When reading action masks, the fixed order of tiered action actions is followed: allow observation, limit frequency, reduce bidding weight, trigger secondary verification, postpone settlement, deduct points, intercept, and upgrade review. Tiered action actions marked with a prohibition flag in the action mask must not be generated as tiered action actions in this step.

[0074] When the policy model generates graded action actions, it first forms a candidate ranking of each graded action action based on the reinforcement learning state vector, and then uses an action mask to remove candidate actions with prohibited markers. When there are tied candidate actions in the candidate ranking, the action that ranks first in the fixed order of graded action actions is given priority. When the first candidate action is prohibited by the action mask, it is postponed to the next candidate action with an allowed marker. When only upgrade review is retained in the action mask, upgrade review is generated. When there is no allowed marker in the action mask, no action action record is generated, and it is registered as a policy check record.

[0075] Action records are written to the action record area of ​​the anti-fraud feature database. The recorded content includes the observation window number, lock record deduplication key, ledger version, state vector rule version, action boundary table version, policy model version, reinforcement learning state vector reference, action mask reference, generated hierarchical action, action effective time, action deduplication key, action deadline, action generation time, and action record version. The reinforcement learning state vector reference and action mask reference point to the submitted records in the state action record area. Plaintext device identifiers, complete Internet addresses, natural person identity fields, and unauthorized review materials are not copied.

[0076] The effective time of an action is the starting time when the generated tiered handling action enters the advertising delivery chain, and this time is included in the action's validity period. The effective time of an action is determined based on the action generation time, the advertising campaign's delivery status, the controllable time slot table of media sources, and the effective delay caliber in the action boundary table. The controllable time slot table of media sources records the batch start time when the advertising delivery chain is allowed to receive frequency limiting, reduced bidding weight, trigger secondary verification, postpone settlement, deduction marking, interception, and upgrade review. When the advertising campaign is in a suspended state, the effective time of the action is written as the first controllable time slot after the next delivery resumes. When the generated tiered handling action is under observation, the effective time of the action can be equal to the action generation time. When the generated tiered handling action involves delivery chain control, the effective time of the action must not be earlier than the most recent controllable time slot in the corresponding controllable time slot table of media sources.

[0077] The action deduplication key is formed by the observation window number, the lock record deduplication key, the ledger version, the state vector rule version, the action boundary table version, the policy model version, the generated hierarchical disposal action, and the action effective time. Only one submitted disposal action record can exist for the same action deduplication key. The result of duplicate policy calls does not overwrite the original disposal action record, but only writes the duplicate action trace. If there is already a disposal action record under the same ledger version with the same lock record deduplication key that has not yet reached its action deadline, no new disposal action record will be generated, unless a new ledger version, state vector rule version, or action boundary table version is submitted in the state action record area.

[0078] The action deadline is the expiration time of the generated tiered action in the advertising delivery chain, and this time is not included in the action's validity period. The action deadline is determined jointly based on the action's effective time, the action's duration definition in the action boundary table version, and the advertising campaign's end time, and must not be later than the advertising campaign's end time. If the action's duration definition exceeds the remaining advertising campaign duration, the advertising campaign's end time will be used as the action deadline. When the generated tiered action is an upgrade review, the action deadline is written to the review waiting boundary. When the generated tiered action is a release observation, the action deadline is written to the end point of the next delivery observation window.

[0079] Action records are appended and cannot be overwritten. If any of the strategy model version, state vector rule version, or action boundary table version changes, only a new action record version can be generated while retaining the previous action record version number. Downstream advertising delivery links read action records according to the action deduplication key, action effective time, action deadline, and generated hierarchical action. Records that have not yet reached their effective time remain in a pending state, and records that have reached their action deadline are no longer read by downstream.

[0080] The security compliance boundary is limited to the fact that the policy model and action record do not access plaintext device identifiers, complete Internet addresses, natural person identity fields and unauthorized review materials. Media sources, advertising slots, device fingerprint clusters and Internet address ranges in the state vector only participate in policy invocation in the form of numbers.

[0081] Preferably, the upper limit of the strategy invocation delay can be set to 5 seconds, the upper limit of the delay for recording the action can be set to 3 seconds, the number of retries for failed strategy invocations can be set to 2, and the retry interval can be set to 2 seconds. For example, the action mask corresponding to a suspicious unconfirmed traffic cluster allows for observation, frequency limiting, reducing bidding weight, triggering secondary verification, suspending settlement, and upgrading review, while prohibiting deduction marking and interception. Under the constraint of this action mask, the strategy model generates a reduction in bidding weight. The action generation time is 10:20:10, the most recently controllable time period is 10:20:15, and the action duration is 30 seconds. If the advertising campaign ends after 10:50:15, the action record will be written with the action effective time of 10:20:15, the action deadline of 10:50:15, and the corresponding deduplication key. Alternatively, in scenarios where the policy model is temporarily unavailable, an offline policy table bound to the same policy model version can be used to generate action records according to the reinforcement learning state vector number and action mask. However, the offline policy table must be locked together with the policy model version, state vector rule version, and action boundary table version, and it still cannot generate graded action prohibiting action masks.

[0082] S6. When both the feedback maturity marker and the feedback credibility marker meet the write-back conditions, a reward value is generated, the strategy model is updated, and the risk boundary, action cooldown condition, and strategy version are written back to the next deployment observation window. The specific implementation is as follows: The write-back process reads the action records in the action record area and the delayed feedback ledger corresponding to the deduplication key of the same locked record in the feedback ledger area. When both the feedback maturity flag and the feedback trust flag meet the write-back conditions, a reward value is generated, the strategy model is updated, and the risk boundary, action cooldown conditions, and strategy version are written back to the next delivery observation window. The write-back process is deployed on the closed-loop learning side of the advertising anti-fraud system. The reading locations include the action record area, feedback ledger area, suspicious record area, risk boundary table release area, action cooldown condition record area, and strategy version record area. The effective period starts after the action record reaches the action deadline and continues until the deadline for the next delivery observation window to receive the write-back record. Action records that have not reached the action deadline do not participate in this step.

[0083] The next deployment observation window is the first deployment observation window after the original deployment observation window to which the current action belongs that has not yet published a risk boundary table and allows the receipt of write-back records; if the deployment observation window has exceeded the deadline for receiving write-back records, it will be postponed to the first deployment observation window that allows the receipt of write-back records.

[0084] The write-back conditions are locked with the write-back rule version. The write-back rule version includes the feedback maturity flag, feedback credibility flag, reward value field definition, reward value range, positive update threshold, negative update threshold, risk boundary adjustment definition, action cooldown condition formation definition, strategy version release definition, and conflict handling definition. When the feedback maturity flag is mature and the feedback credibility flag is high credibility, the write-back conditions are directly met. When the feedback maturity flag is mature and the feedback credibility flag is medium credibility, only a low-weight write-back record is generated. When the feedback maturity flag is immature, partially mature, or the feedback credibility flag is low credibility, no reward value is generated, and it is registered as a write-back record pending verification. Low-weight write-back records are write-back records that generate reward values ​​using a low-weight scale and do not directly replace the published risk boundary table. They only participate in the formation of pending risk boundaries, action cooldown conditions, and strategy update records.

[0085] The reward value is an integer feedback record formed around the same locked record deduplication key, and the value range is locked according to the write-back rule version. When generating the reward value, the hierarchical disposal action, action effective time, action deduplication key, and action deadline generated in the disposal action record are read. The immediate feedback field, conversion feedback field, review feedback field, feedback maturity mark, and feedback trust mark in the delayed feedback ledger are read. The risk value, suspicious lower boundary, and interception upper boundary in the suspicious status record are also read.

[0086] The reward value fields are formed in a fixed order according to the write-back rules version. The fixed order is: valid conversion retention field, invalid traffic confirmation field, consumption saving field, normal traffic loss field, review withdrawal field, and action cost field. The valid conversion retention field, invalid traffic confirmation field, and consumption saving field are obtained as integer scores according to the positive scale in the write-back rules version, while the normal traffic loss field, review withdrawal field, and action cost field are obtained as integer scores according to the negative scale in the write-back rules version. When the feedback credibility mark is medium credibility, each field is obtained as an integer score according to the low-weight scale, and the high-credibility scale is not used. Among them, the consumption saving field is formed based on the unbilled amount, frozen amount, and deducted amount associated with the same action deduplication key in the settlement side status record. The amount that is not associated with the action deduplication key is not included in the consumption saving field. The normal traffic loss field is formed based on the record of invalid judgment revocation in the review feedback field and the number of conversion feedback fields that were affected by frequency limiting, reduced bidding weight, suspended settlement, deduction mark, or interception from the action effective time to the action deadline and were subsequently authorized for review and confirmed as valid.

[0087] After each field obtains an integer score, the reward value is accumulated in a fixed field order. If the accumulated result exceeds the positive upper limit in the write-back rule version, it is recorded at the positive upper limit; if it is lower than the negative lower limit, it is recorded at the negative lower limit. If any of the following is missing: field source, conflicting review conclusion, missing action effective time, action deduplication key, or action deadline, the generation of reward value is stopped and the record is registered as a reward pending verification record.

[0088] When updating the policy model, the write-back process first writes the reinforcement learning state vector reference, action mask reference, generated hierarchical action, reward value, policy model version, and write-back rule version to the policy update record area. Only after the policy update record area is fully logged can a new policy version be created; the old policy model version must not be overwritten. When creating a new policy version, the write-back process assigns the reinforcement learning state vector to the corresponding state interval according to the state interval number in the policy model version. When the reward value reaches the positive update threshold, the candidate priority of the hierarchical action generated in that state interval is increased by 1 level. When the reward value reaches the negative update threshold, the candidate priority of the hierarchical action generated in that state interval is decreased by 1 level. When the reward value does not reach either the positive or negative update threshold, only the policy update record is written, without changing the candidate priority. Candidate priority adjustments must not allow prohibited hierarchical actions in the action mask to enter the selectable range.

[0089] New strategy versions are appended to the strategy version record area. The strategy version record includes the previous strategy model version, the new strategy version, the update batch number, the update time, the deduplication key set of the lock records involved in the update, the write-back rule version, and the pending or active status. When the write-back rule version requires batch release, the new strategy version is first written to the pending status, and then written to the active status after all strategy update records in the same update batch have been recorded.

[0090] When writing back the risk boundary to the next delivery observation window, the write-back process reads the risk value, suspicious lower boundary, interception upper boundary, and reward value from the current suspicious status record, and forms the risk boundary for the next delivery observation window according to the boundary step size table in the write-back rule version. When the reward value reaches the positive write-back threshold and there are no normal traffic loss fields or review withdrawal fields, the suspicious lower boundary and interception upper boundary under the same media source, ad slot, device fingerprint cluster formation source, and Internet address range source are each lowered by 1 boundary step size, so that subsequent traffic clusters of the same observation window enter the suspicious and interception range in a lower risk value range. When the reward value reaches the negative write-back threshold or there are normal traffic loss fields or review withdrawal fields, the risk boundary of the same type is each increased by 1 boundary step size, so that subsequent traffic clusters of the same observation window enter the suspicious and interception range in a higher risk value range. The suspicious lower boundary after the downward or upward adjustment must be smaller than the interception upper boundary and must not exceed the minimum and maximum boundaries in the write-back rule version. If boundary crossing or exceeding occurs after adjustment, the risk boundary will not be published and will be registered as a risk boundary pending verification record.

[0091] When multiple write-back records simultaneously affect the risk boundary under the same media source, advertising space, device fingerprint cluster formation source, or Internet address range source, positive and negative write-back records within the same write-back batch are first merged. If the number of positive write-back records is greater than the number of negative write-back records, the risk boundary is adjusted downwards; if the number of negative write-back records is greater than the number of positive write-back records, the risk boundary is adjusted upwards. If the number of review withdrawal fields is the same or there are conflicting fields, the risk boundary is not adjusted, and only the write-back traces are retained.

[0092] When writing back the action cooling conditions to the next delivery observation window, the write-back process forms the cooling object, cooling action type, cooling start point, cooling end point, and action cooling condition version based on the generated tiered action, action effective time, action end time, reward value, and review feedback fields. The cooling object is jointly determined by the media source, ad slot, device fingerprint cluster, and Internet address range. The cooling action type is the generated tiered action. The cooling start point is the action end time. The cooling end point is formed according to the cooling duration table in the write-back rule version.

[0093] If the same cooling object and the same cooling action type have cooling conditions that have not yet reached the cooling endpoint in the next deployment observation window, the later cooling endpoint shall be taken as the new cooling endpoint; if the reward value reaches the negative write-back threshold or the review withdrawal field appears, the cooling endpoint shall be extended according to the write-back rule version; if the reward value reaches the positive write-back threshold and there is no normal traffic loss field, the cooling endpoint shall be shortened according to the write-back rule version, but it shall not be earlier than the start of the next deployment observation window.

[0094] The risk boundary version is formed by the next deployment observation window number, the write-back rule version, and the risk boundary adjustment batch number; the action cooldown condition version is formed by the next deployment observation window number, the write-back rule version, and the action cooldown condition batch number; risk boundaries and action cooldown conditions within the same version cannot be mixed across write-back rule versions; when writing back the strategy version to the next deployment observation window, the write-back process writes the new strategy version, risk boundary version, and action cooldown condition version into the write-back record area of ​​the next deployment observation window. The record content includes the next deployment observation window number, lock record deduplication key, action deduplication key, reward value, write-back rule version, new strategy version, risk boundary version, action cooldown condition version, write-back time, write-back record deduplication key, and effective status; the write-back record deduplication key is formed by the next deployment observation window number, lock record deduplication key, action deduplication key, write-back rule version, new strategy version, risk boundary version, and action cooldown condition version. Only one submitted write-back record can exist for the same write-back record deduplication key. Duplicate write-back results do not overwrite the original record, but only write the duplicate write-back trace.

[0095] When reading the next deployment observation window, the write-back records that have been submitted and reached the effective state will be used as the basis; new strategy versions, risk boundaries, and action cooldown conditions that have not reached the effective state will not be included in the next deployment observation window.

[0096] The security compliance boundary is limited to the fact that the write-back process does not write plaintext device identifiers, complete Internet addresses, natural person identity fields, or unauthorized review materials. The reward value and write-back record only save the number of times, amount, action, version, and time after being associated with the deduplication key of the locked record.

[0097] Preferably, the reward value can be set to an integer from -1000 to +1000, the positive write-back threshold can be set to 300, the negative write-back threshold can be set to -300, the single boundary step size can be set to 20, the upper limit of the write-back batch delay can be set to 60 seconds, the sample size for sampling before the strategy version is released can be set to no less than 100 records per batch, the number of retries for write-back failure can be set to 3, and the retry interval can be set to 10 seconds; for example, the action record for handling a suspicious unconfirmed traffic cluster is to reduce the bidding weight, the action takes effect at 10:20:15, the action ends at 10:50:15, the feedback maturity is marked as mature, the feedback credibility is marked as highly credible, and the conversion feedback field shows that it is retained within the validity period of the action. After 320 conversions, the review feedback field confirmed that the corresponding traffic was in an invalid traffic state and there was no review withdrawal field. The write-back process generated a reward value of 680. This reward value reached the positive write-back threshold and there was no normal traffic loss field. Therefore, a new strategy version was formed, and the risk boundary of the same type was lowered by 1 boundary step. At the same time, the corresponding action cooling conditions and strategy version were written back to the next deployment observation window. Alternatively, in scenarios where the strategy model adopts a periodic release method, this step can first generate a strategy version to be effective, and then set the strategy version to be effective to the effective state after the same write-back batch is completed. However, the risk boundary, action cooling conditions, and strategy version must still be associated with the next deployment observation window according to the same write-back rule version.

[0098] In the operating scenario shown in this embodiment: the advertiser places advertising campaign A in the advertising anti-fraud system. Advertising campaign A is continuously placed in advertising slot B of media source A. The system defines 09:00:00 to 09:15:00 as the current placement observation window. The window start time of 09:00:00 is included in the current placement observation window, and the window end time of 09:15:00 is not included in the current placement observation window. Events arriving at 09:15:00 are included in the unified placement observation window.

[0099] The advertising anti-fraud system receives exposure logs from the media traffic access layer, request logs from the advertising request service, click logs from the click redirect service, access logs from the landing page tracking service, conversion logs from the conversion feedback service, consumption logs from the settlement side, and traffic quality feedback logs within the current ad placement observation window, and completes the collection of data in the current ad placement observation window before the confirmation deadline of 09:18:00.

[0100] The system first performs time windowing based on the server's reception time, then merges the media source number, application package name, site domain name, and channel mark into media source A through the media access configuration table, and confirms ad position B through ad position number, page position, display style, and ad position version; it forms device fingerprint cluster C for the same encrypted device identifier, and forms internet address segment D by organizing the requested internet address through the address segment rule table.

[0101] After field denoising and field completion within the allowed range, events where media source A, ad placement B, device fingerprint cluster C, and internet address range D are completely identical within the current ad viewing window are aggregated into one viewing window traffic cluster record. This viewing window traffic cluster record includes viewing window number W0900, media source A, ad placement B, device fingerprint cluster C, internet address range D, 120,000 impressions, 6,400 clicks, 5,100 landing page visits, 380 conversion feedbacks, cost, and the record deduplication key K-W0900-A-B-C-D, and is appended to the viewing window record area of ​​the anti-fraud feature database.

[0102] After the anti-fraud judgment process reads the traffic cluster records of the observation window, it matches the risk boundary table version R1. In this risk boundary table version, the lower boundary of suspicion is 420, and the upper boundary of interception is 760. The risk value is formed by taking integer scores in the order of fixed fields: the degree of aggregation of clicks relative to exposure, the degree of acceptance of landing page visits relative to clicks, the degree of deviation of conversion feedback relative to clicks, the consumption intensity of consumption amount relative to exposure, the degree of concentration of fingerprint clusters of the same device, and the degree of concentration of the same Internet address segment.

[0103] The system generates a risk value of 610 according to the fixed field order. Since the risk value of 610 reaches the suspicious lower boundary of 420 and is lower than the interception upper boundary of 760, the traffic cluster record in the observation window is locked as a suspicious unconfirmed traffic cluster. The suspicious unconfirmed traffic cluster record is written in the suspicious record area, and the lock time is recorded as 09:16:00, the risk boundary table version is R1, and the deduplication key of the locked record is K-L0900-001.

[0104] The anti-fraud ledger process then establishes a delayed feedback ledger based on the deduplication key K-L0900-001 of the locked record. It first writes the immediate feedback fields, including impressions (120,000), clicks (6,400), landing page visits (5,100), cost, initial traffic quality feedback status, and current status on the settlement side. Within 24 hours of the locked time, the conversion feedback service returns 380 conversion feedbacks. The ledger process appends the conversion feedback number, conversion feedback time, conversion type, conversion confirmation source, and conversion withdrawal status to the conversion feedback fields. Within 48 hours of the locked time, the authorized review source returns a review record, confirming that the suspicious unconfirmed traffic cluster has an invalid traffic status and no review withdrawal has occurred.

[0105] The ledger process confirms that the immediate feedback fields are complete, there are no conversion records pending verification within the feedback deadline, and there is an authorized review conclusion within the review waiting boundary, forming a maturity marker; at the same time, it confirms that the immediate feedback fields, conversion feedback fields, and review feedback fields all come from authorized sources with version numbers, the association keys are complete, and there are no late conversion traces, duplicate review traces, or review conflict traces, forming a high-confidence marker.

[0106] The status processing process reads suspicious status records and the delayed feedback ledger, associates the two according to the deduplication key K-L0900-001 of the locked record, and organizes the risk value 610, boundary interval, complete number of immediate feedback, number of arrival of conversion feedback, number of verification feedback, number of mature feedback, number of reliable feedback, number of locked duration, number of media source, number of ad slot, number of device fingerprint cluster, and number of Internet address range into a fixed-length reinforcement learning state vector according to the status vector rule version S1. Then, an action mask is generated according to the action boundary table version A1. The action mask allows observation, frequency limiting, reduction of bidding weight, triggering of secondary verification, suspension of settlement and upgrade verification, and prohibits deduction marking and interception. The system limits the scope of graded handling actions accordingly and writes them into the status action record area.

[0107] The policy invocation process reads the reinforcement learning state vector and action mask, and submits them to the policy model version M1. The policy model version M1 forms a candidate sort according to the order of state vector reading and action candidate score scale, and is constrained by the action mask to exclude deduction marks and interception, and finally generates a reduction in bidding weight as a graded disposal action.

[0108] The policy invocation process writes a handling action record in the handling action record area, recording the action effective time 10:20:15, the action deduplication key K-ACT-0900-001, the action deadline 10:50:15, the policy model version M1, the action boundary table version A1, and the action record version.

[0109] After the action deadline is reached, the write-back process reads the action record and the delayed feedback ledger corresponding to the deduplication key of the same locked record. It confirms that the feedback maturity mark is mature and the feedback credibility mark is highly credible, and generates a reward value according to the write-back rule version B1. Among them, the effective conversion retention field, invalid traffic confirmation field, and consumption saving field obtain positive scores, while the normal traffic loss field, review withdrawal field, and action cost field obtain corresponding negative scores. After accumulation, a reward value of 680 is formed.

[0110] Since the reward value of 680 reaches the positive write-back threshold of 300, and there are no normal traffic loss fields or review withdrawal fields, the write-back process raises the priority of the candidate with reduced bidding weight within the state interval corresponding to the reinforcement learning state vector by 1 level, forming a new strategy version M2; at the same time, the suspicious lower boundary and interception upper boundary under the same media source A, ad position B, device fingerprint cluster formation source, and Internet address range source are each lowered by 1 boundary step, forming the risk boundary version R2 of the next delivery observation window, and the action cooling condition version C1 is generated according to the action cooling condition formation caliber, with the action deadline time of 10:50:15 as the cooling start point.

[0111] The next delivery observation window is the first delivery observation window after the original delivery observation window that has not yet published a risk boundary table and allows receiving write-back records. The write-back process writes the new strategy version M2, risk boundary version R2, action cooling condition version C1, and reward value 680 into the write-back record area of ​​the next delivery observation window. When the next delivery observation window starts, it only reads the write-back records that have been submitted and have reached the effective state, so that the traffic clusters of the same type of observation window enter the suspicious range and interception range in the lower risk value range, and avoids generating the same graded handling action repeatedly for the same cooling object before the action cooling condition reaches the cooling end.

[0112] To verify the impact of the delayed feedback ledger, action mask, reward value, and next delivery observation window write-back mechanism in this embodiment on the handling of suspicious and unconfirmed advertising traffic, a sample data of advertising event chain with the same field structure as this embodiment was constructed for replay verification. This sample data includes exposure logs, request logs, click logs, landing page access logs, conversion feedback logs, consumption logs, and traffic quality feedback logs. During replay, the advertising campaign identifier, media source, ad placement, device fingerprint cluster, internet address range, initial risk boundary table, and initial strategy model version remained consistent. The comparison scheme uses an existing rule threshold scheme, directly generating a fixed handling action after the risk value reaches the preset handling boundary; the scheme in this embodiment sequentially forms the observation window traffic cluster record, suspicious and unconfirmed traffic cluster, delayed feedback ledger, reinforcement learning state vector, action mask, handling action record, reward value, and next delivery observation window write-back record according to S1 to S6.

[0113] In a replay verification, sample data from advertising event chains within 12 consecutive observation windows were used as the verification object, resulting in 2400 traffic cluster records. Among these, 360 records were classified as suspicious unconfirmed traffic clusters with risk values ​​reaching the suspicious lower boundary but below the interception upper boundary. The verification benchmark was set according to the field definitions of the review feedback field and traffic quality feedback record in this embodiment. The existing rule threshold scheme generated action actions for 214 suspicious unconfirmed traffic clusters, of which 163 records were confirmed as invalid traffic, 36 records were confirmed as valid traffic, and the remaining records were not confirmed. The scheme in this embodiment generated action actions for 228 suspicious unconfirmed traffic clusters, of which 197 records were confirmed as invalid traffic, 17 records were confirmed as valid traffic, and the remaining records were not confirmed. The interception accuracy rate is calculated based on the proportion of traffic subsequently identified as invalid within the processed traffic cluster. The interception accuracy rate of the existing rule threshold scheme is 76.2%, while the interception accuracy rate of the scheme in this embodiment is 86.4%. The false positive rate is calculated based on the proportion of traffic subsequently identified as valid within the processed traffic cluster. The false positive rate of the existing rule threshold scheme is 16.8%, while the false positive rate of the scheme in this embodiment is 7.5%.

[0114] In the above replay verification, the budget savings were calculated based on the unbilled amount, frozen amount and deducted amount associated with the action deduplication key. The budget savings rate formed by the existing rule threshold scheme was 8.1%, while the budget savings rate formed by the above example scheme in this implementation was 13.6%.

[0115] Table 1 shows the comparison results of playback verification between the existing rule threshold scheme and the scheme in this embodiment.

[0116] Table 1. Comparison of playback verification results between existing rule-based threshold schemes and the scheme in this embodiment:

[0117] The interception accuracy rate is calculated based on the percentage of records in the processed traffic cluster that are subsequently confirmed as invalid traffic; the false positive rate is calculated based on the percentage of records in the processed traffic cluster that are subsequently confirmed as valid traffic; and the budget saving rate is calculated as the ratio of the sum of unbilled amount, frozen amount, and deducted amount associated with the same action deduplication key to the baseline consumption amount of the same advertising campaign. Because this embodiment continues to read the delayed feedback ledger after generating the action record and generates a reward value when both the feedback maturity flag and the feedback credibility flag meet the write-back conditions, it can incorporate subsequent conversion feedback, review feedback, and settlement-side status records into the risk boundary version, action cooling condition version, and strategy version update process of the next delivery observation window.

[0118] Statistical analysis of the label distribution of delayed feedback ledgers revealed that, among the 360 ​​suspected unconfirmed traffic cluster records, 287 ledgers (79.7%) had mature labels, 51 ledgers (14.2%) had partially mature labels, and 22 ledgers (6.1%) had immature labels. Regarding feedback credibility labels, 219 ledgers (60.8%) had high-credibility labels, 93 ledgers (25.8%) had medium-credibility labels, and 48 ledgers (13.4%) had low-credibility labels.

[0119] The distribution results of feedback maturity markers and feedback credibility markers in the aforementioned delayed feedback ledger are shown in Table 2.

[0120] Table 2: Distribution of feedback markers in the delayed feedback ledger:

[0121] As shown in Table 2, the delayed feedback ledger can record feedback maturity and credibility in a hierarchical manner based on the immediate feedback field, the conversion feedback field, the review feedback field, and the feedback source status, thereby providing differentiated feedback input for reward value generation.

[0122] The version evolution within the continuous campaign observation window was statistically analyzed. The initial observation window used risk boundary version R1, action cooldown condition version C1, and strategy model version M1. After the fourth observation window, due to the accumulated positive reward value from action records under the same media source, same ad placement, same device fingerprint cluster, and same internet address range, the write-back process prioritized the tiered action candidates generated within the corresponding state interval, resulting in strategy model version M2, risk boundary version R2, and action cooldown condition version C2. After the eighth observation window, due to the presence of normal traffic loss fields in some action records, the write-back process extended the corresponding action cooldown conditions according to the write-back rule version, resulting in action cooldown condition version C3. After the twelfth observation window, the strategy model version was updated to M3, and the risk boundary version was updated to R3. Within the 12 consecutive observation windows, the number of times the same error handling was repeatedly triggered under the same media source, same ad placement, same device fingerprint cluster, and same internet address range decreased from 19 times in the first four observation windows to 7 times in the last four.

[0123] Table 3 shows the version evolution and changes in handling duplicate errors in the continuous delivery observation window.

[0124] Table 3. Version evolution and effect changes in the continuous deployment observation window:

[0125] As shown in Table 3, as feedback maturity markers, feedback credibility markers, reward values, and write-back records continue to form in the continuous delivery observation window, the risk boundary version, action cooling condition version, and strategy model version are adjusted sequentially. The number of times the same error handling is repeatedly triggered under the same media source, the same ad slot, the same device fingerprint cluster, and the same Internet address range decreased from 19 times in the first 4 delivery observation windows to 7 times in the last 4 delivery observation windows.

[0126] Based on the replay verification results in Tables 1 to 3, it can be seen that the solution in this embodiment does not simply rely on a single risk value within the current observation window for fixed handling. Instead, after a suspicious unconfirmed traffic cluster is locked, it uses a delayed feedback ledger to incorporate the immediate feedback field, conversion feedback field, and review feedback field, and utilizes feedback maturity and feedback credibility markers to determine the reward value generation conditions. Then, the reward value drives the strategy model version, risk boundary version, and action cooldown condition version to be written into the next delivery observation window. This allows subsequent similar advertising traffic clusters to access the previously written version records when entering the suspicious range, triggering tiered handling actions, or initiating action cooldown conditions. These replay verification results demonstrate the technical effectiveness of this embodiment in reducing the false positive rate of suspicious unconfirmed traffic clusters, improving the accuracy of invalid traffic handling, and reducing repeated erroneous handling under similar sources, similar ad placements, similar device fingerprint clusters, and similar internet address ranges.

[0127] All calculations involved in the embodiments are dimensionless numerical calculations, and the preset parameters and thresholds in the calculations are set by those skilled in the art according to the actual situation.

[0128] It should be noted that this invention can be deployed on the device itself to realize embedded applications, or it can run on a PC or other terminal with a user interface, thereby meeting various hardware environments and usage requirements.

[0129] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wireless or wired transmission; wired transmission methods include optical fiber, twisted pair, coaxial cable, etc.; wireless transmission includes infrared, microwave, etc. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center containing one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0130] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0132] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0133] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0134] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0135] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0136] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for combating advertising traffic fraud based on reinforcement learning, characterized in that, include: S1. Collect advertising event chain data according to the placement observation window, and aggregate it into observation window traffic cluster records based on media source, ad position, device fingerprint cluster, and Internet address range; S2. Match the traffic cluster records in the observation window to the risk boundary table, and lock the suspicious unconfirmed traffic clusters whose risk values ​​reach the suspicious lower boundary and are lower than the interception upper boundary. S3. Establish a delayed feedback ledger for suspicious and unconfirmed traffic clusters, write the instant feedback field, conversion feedback field, and review feedback field, and form feedback maturity mark and feedback credibility mark; S4. Generate reinforcement learning state vectors based on suspicious state records and delayed feedback ledgers, and generate action masks based on action boundary tables to limit graded handling actions. S5. Submit the reinforcement learning state vector and action mask to the policy model to generate a disposal action record, and write the action effective time, action deduplication key, and action deadline into the disposal action record. S6. When both the feedback maturity mark and the feedback credibility mark meet the write-back conditions, generate a reward value, update the strategy model, and write back the risk boundary, action cooldown conditions, and strategy version to the next deployment observation window.

2. The method for combating advertising traffic fraud based on reinforcement learning according to claim 1, characterized in that, S1 includes: The server receiving time is used as the event window time for advertising event chain data; Time is assigned to windows according to the following rules: the starting point of the window is included in the current delivery observation window, the ending point of the window is not included in the current delivery observation window, and events falling at the ending point of the window are assigned to the next delivery observation window. The criteria for identifying duplicates are formed by event number, advertising campaign identifier, media source, advertising space, device fingerprint cluster, Internet address range, and event window time. Events with consistent media sources, ad placements, device fingerprint clusters, internet address ranges, and fields are aggregated into observation window traffic cluster records in the order of time windowing, source matching, ad placement matching, device fingerprint clusters, internet address ranges, field denoising, and field completion.

3. The method for anti-fraud of advertising traffic based on reinforcement learning according to claim 1, characterized in that, S2 include: Read the submitted traffic cluster records in the observation window that have been aggregated and marked as complete; Confirm the applicable start and end times of the risk boundary table based on the observation window number; Risk boundary tables are matched based on advertising campaign identifiers, media sources, ad placements, device fingerprint cluster formation sources, and internet address range sources; Risk values ​​are generated based on a fixed field order and a fixed scale for forming the risk value definition; When the risk value reaches the lower boundary of suspicion but is below the upper boundary of interception, the corresponding suspicious unconfirmed traffic cluster record will be written into the suspicious record area.

4. The method for anti-fraud of advertising traffic based on reinforcement learning according to claim 1, characterized in that, S3 include: Create a delayed feedback ledger in the feedback ledger area by using the observation window number, the locked batch number, and the deduplication key for the locked record; According to the feedback rules, the feedback deadline, the review waiting boundary, and the list of authorized sources are locked. Add immediate feedback, conversion feedback, and review feedback fields to the delayed feedback ledger; Based on field gap markers, converted records awaiting verification, feedback deadlines, review waiting boundaries, and review conflict logs, feedback maturity markers are generated. Based on the list of authorized sources, the complete status of associated keys, and the verification of conflict traces, a reliable feedback marker is generated.

5. The method for anti-fraud of advertising traffic based on reinforcement learning according to claim 1, characterized in that, S4 include: Deduplicat the locked record and associate it with the suspicious status record and the delayed feedback ledger; Read the risk value, lower boundary of suspicion, upper boundary of interception, lock time, media source, ad slot, device fingerprint cluster and Internet address range from the suspicious status record; Read the immediate feedback field, conversion feedback field, review feedback field, feedback maturity flag, and feedback credibility flag from the delayed feedback ledger; A fixed-length reinforcement learning state vector is formed by following the fixed field order of the state vector rule version.

6. The method for combating advertising traffic fraud based on reinforcement learning according to claim 5, characterized in that, S4 also includes: Read the effective action boundary table and match the action boundary table according to media source, ad position, feedback maturity flag, feedback credibility flag, risk value range and lock duration range; Generate an action mask based on a fixed sequence of graded handling actions; Exclude actions corresponding to prohibited tags based on action masks, and retain actions corresponding to allowed tags based on action masks; The reinforcement learning state vector, action mask, and the defined hierarchical handling actions are written into the state-action record area.

7. The method for anti-fraud of advertising traffic based on reinforcement learning according to claim 1, characterized in that, S5 include: Read the submitted reinforcement learning state vector, action mask, and limited hierarchical processing actions from the state-action record area; The candidate order for each level of action is formed by reading the state vectors fixed in the strategy model version and the action candidate scoring scale; Eliminate tiered actions with prohibited markers in the action mask; Generate a record of actions that includes the first-ranked action in the remaining candidate sorting, and write the action's effective time, deduplication key, and deadline.

8. The method for anti-fraud of advertising traffic based on reinforcement learning according to claim 1, characterized in that, S6 include: Read the action record and delay feedback ledger after the action deadline has passed; The write-back rules are used to lock the write-back conditions, the definition of the reward value field, the positive update threshold, the negative update threshold, and the policy version release definition. A reward value is generated when both the feedback maturity mark and the feedback trust mark meet the write-back conditions. The reinforcement learning state vector reference, action mask reference, generated hierarchical disposal action, reward value, policy model version, and write-back rule version are written into the policy update record area, and a new policy version is formed by appending.

9. The method for combating advertising traffic fraud based on reinforcement learning according to claim 8, characterized in that, S6 also includes: Based on the reward value, risk value, suspicious lower boundary, and interception upper boundary, a risk boundary version is formed according to the boundary step table in the write-back rule version; Based on the generated graded handling actions, action effective time, action deadline, reward value, and review feedback fields, an action cooldown condition version is formed; Write the new strategy version, risk boundary version, and action cooldown condition version to the write-back record area of ​​the next deployment observation window.

Images