Buried point multiple detection method and device, storage medium and computer equipment

Abstract

The method, apparatus, storage medium, and computer equipment for detecting multiple event tracking points provided in this application include: acquiring event tracking point log data to be detected; grouping the event tracking point log data according to release version, business platform, terminal identifier, and date; sorting the event tracking point log data within each comparison group according to the event tracking point sending time; comparing the event tracking point log data within each comparison group by using the first event tracking point log data as the current starting point of a sliding window and a preset time length as the length of the sliding window; recording duplicate event tracking point log data based on the comparison results; and moving the current starting point of the sliding window one event tracking point log data to the right, continuing the comparison until the current starting point of the sliding window reaches the last event tracking point log data in the comparison group. This automatically locates duplicate event tracking point log data.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, storage medium, and computer equipment for detecting multiple embedded points. Background Technology

[0002] As mobile applications become increasingly feature-rich, they typically collect page behavior and business data through event tracking for subsequent statistical analysis. In actual development, event tracking logic is often distributed across different functional modules and interface logic to record page displays, business object exposures, and related interactions, thereby providing foundational data support for BI (Business Intelligence) systems.

[0003] However, during application development, multiple event tracking points can easily occur. For example, on the same page, due to the need to request multiple APIs, a UI component on the page may be refreshed multiple times after asynchronous data is returned, resulting in the corresponding event tracking point being triggered repeatedly. Similarly, in list-type pages, product exposure event tracking points are usually triggered during scrolling; due to view reuse or refresh mechanisms, the same product slot may be reported repeatedly. These situations are not multiple real behaviors in business logic, but rather duplicate data collection caused by the implementation method.

[0004] Duplicate event tracking consumes additional network and storage resources, increasing system load. Furthermore, it interferes with BI data analysis results, causing deviations in statistical indicators such as exposure and reducing the accuracy of data analysis. Therefore, identifying and controlling such frequent event tracking issues has become a practical technical requirement for ensuring the quality of event tracking data. Summary of the Invention

[0005] The purpose of this application is to at least address one of the aforementioned technical deficiencies, particularly the technical deficiencies in the prior art regarding how to identify and control the frequent occurrence of such embedded points.

[0006] Firstly, this application provides a method for detecting multiple embedded points, the method comprising:

[0007] Obtain the log data of the event tracking points to be detected;

[0008] The log data of each monitoring point to be tested is grouped according to the release version, business platform, terminal identifier and date to obtain each comparison group. Within each comparison group, the log data of each monitoring point to be tested is sorted according to the sending time of the monitoring point.

[0009] Within each sorted comparison group, the first log data to be detected is used as the current starting point of the sliding window, and the preset time length is used as the length of the sliding window. The log data to be detected within the sliding window are compared. Based on the comparison results, duplicate log data are recorded, and the current starting point of the sliding window is moved backward by one log data to be detected. The comparison continues until the current starting point of the sliding window moves to the last log data to be detected within the comparison group.

[0010] In one embodiment, after the step of obtaining the log data of the event tracking point to be detected, the following steps are included:

[0011] Based on a preset blacklist, the log data of each monitoring point to be detected is filtered.

[0012] In one embodiment, the step of grouping the log data of each monitoring point to be detected according to the release version, business platform, terminal identifier, and date to obtain each comparison group includes:

[0013] Based on a preset terminal identifier whitelist, the log data corresponding to the terminal identifiers in the log data to be detected are filtered out, and then grouped according to the release version, business platform, terminal identifier and date to obtain each comparison group.

[0014] In one embodiment, the step of comparing the log data of the event tracking points to be detected within the sliding window and recording duplicate event tracking point log data based on the comparison result includes:

[0015] Based on the key business field content in each log data point to be detected within the sliding window, determine the summary identifier of each log data point to be detected.

[0016] Each log data point to be detected within the sliding window is compared pairwise according to its corresponding summary identifier. Log data points with the same summary identifier are recorded as duplicate log data points.

[0017] In one embodiment, the step of determining the summary identifier of each log data point to be detected based on the content of key business fields in each log data point to be detected within the sliding window includes:

[0018] After replacing the dynamic variables that conform to the preset rules in the key business fields of each log data point to be detected within the sliding window with static unified values, a summary identifier for each log data point to be detected is generated using a preset summary algorithm.

[0019] In one embodiment, the step of moving the current starting point of the sliding window backward by one log data point to be detected and continuing the comparison includes:

[0020] The next event log data to be detected at the current starting point of the sliding window is taken as the target event log data, and it is determined whether the target event log data has been recorded as duplicate event log data.

[0021] If not, the target event log data will be used as the new starting point of the sliding window, and the comparison will continue.

[0022] If so, the target event log data will no longer be used as the current starting point of the sliding window, and the next event log data to be detected will be used as the new target event log data to continue to determine whether it has been recorded as duplicate event log data.

[0023] In one embodiment, the method further includes:

[0024] For two groups to be compared on adjacent natural days, obtain the log data of the monitoring point to be detected in the previous group to be compared whose sending time is later than the first threshold time point, and the log data of the monitoring point to be detected in the next group to be compared whose sending time is earlier than the second threshold time point, to form a cross-day detection set.

[0025] Within the cross-day detection set, the log data of each detection point is sorted according to the sending time of the detection point, and duplicate detection point log data is identified and recorded.

[0026] Secondly, this application provides a device for detecting multiple embedded points, the device comprising:

[0027] The module for acquiring log data of the event tracking points to be detected is used to acquire log data of the event tracking points to be detected.

[0028] The comparison group determination module is used to group the log data of each monitoring point according to the release version, business platform, terminal identifier and date to obtain each comparison group, and sort the log data of each monitoring point according to the sending time of the monitoring point within each comparison group.

[0029] The duplicate event log data recording module is used to compare the event log data within each sorted comparison group, using the first event log data to be detected as the current starting point of the sliding window and a preset time length as the length of the sliding window. Based on the comparison result, duplicate event log data is recorded, and the current starting point of the sliding window is moved backward by one event log data to be detected. The comparison continues until the current starting point of the sliding window moves to the last event log data to be detected in the comparison group.

[0030] Thirdly, this application provides a storage medium storing computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of any of the above-described methods for detecting multiple embedded points.

[0031] Fourthly, this application provides a computer device, including: one or more processors, and a memory;

[0032] The memory stores computer-readable instructions, which, when executed by one or more processors, perform the steps of any of the above-described methods for detecting multiple embedded points.

[0033] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0034] The method, apparatus, storage medium, and computer equipment for detecting multiple instances of event tracking provided in this application first group the logs according to release version, business platform, terminal identifier, and date. This confines the originally mixed full logs to a comparable range within the same version, environment, terminal, and time dimension, eliminating the interference of cross-version differences, platform differences, and user differences on the judgment results and establishing a unified data benchmark for repeatability determination. Second, within each group, the logs are sorted by sending time to construct a time sequence of actual behavior, giving the event tracking trigger relationship a traceable temporal logical basis. Furthermore, by setting a sliding window of fixed time length, the event tracking logs within a local time interval are compared window by window. This can capture dense reporting behavior caused by repeated interface calls, repeated UI refreshes, or abnormal exposure logic within a short period of time, which can more accurately characterize the time-related feature of multiple instances compared to global statistical methods. At the same time, the sliding window slides sequentially according to the log granularity to achieve complete coverage of the entire sequence and ensure detection integrity. Based on this, this application can not only automatically locate duplicate event log data and identify abnormal high-frequency triggering patterns, but also provide data basis for subsequent event deduplication, rule optimization and trigger logic rectification, thereby reducing the amount of invalid data, saving transmission and storage resources, improving the authenticity and usability of event data, and ultimately achieving effective identification and controllable management of the problem of frequent event occurrences. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 A flowchart illustrating the multiple detection method for embedded points provided in this application embodiment;

[0037] Figure 2 This is a schematic diagram of the structure of the embedded point multiple detection device provided in the embodiments of this application;

[0038] Figure 3 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0039] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] This application provides a method for detecting multiple data entry points. The following embodiments illustrate this method using a computer device as an example. It is understood that the computer device can be any device with data processing capabilities, including but not limited to a single server, server cluster, personal laptop, desktop computer, etc. Figure 1 As shown, the method may include the following steps:

[0041] S101: Obtain the log data of the event tracking point to be detected.

[0042] Among them, the log data of the event to be detected refers to the set of recorded information generated and reported by the terminal device or business system around user behavior, interface state changes or function call events during the operation of the application. The log data includes at least event identification information, trigger time information and context data related to the triggering scenario, which are used to reflect the behavioral semantics and environmental state when an event is triggered.

[0043] In real-world applications, applications trigger event tracking and reporting when pages load, API responses are received, elements are exposed, or user actions occur. Different terminals continuously generate a large number of log records at different times, which are typically transmitted over the network to a backend log receiving service. To implement this, a unified log access channel can be established in the server-side runtime environment. This channel continuously receives event tracking and reporting requests from multiple terminals via network monitoring. The received data is then parsed according to a predetermined log structure, ensuring that the event tracking identifier, timestamp, and scenario field in each log record are correctly separated and written to a memory cache or temporary storage area. When dealing with large volumes of logs, historical log files can be periodically read in batches from existing log storage media. The event tracking records in these files are parsed line by line and loaded into a processing queue, thus ensuring compatibility with both real-time data streams and offline logs, guaranteeing the integrity of the data source.

[0044] During the data access process, to ensure the accuracy of subsequent detection results, basic validity checks can be performed on the acquired log data. These checks include verifying the completeness of the log format, the presence of the time field, and whether the tracking identifier is empty. Clearly abnormal or corrupted records are marked or removed during the access phase, ensuring structural consistency of the data entering the detection process. Simultaneously, access time information or source identifier information can be appended to each log entry, giving the same record traceability in subsequent processing. This allows for differentiation of data sets from different sources even in scenarios with multiple terminals and versions running in parallel. Through these methods, tracking logs from different terminals and business processes are uniformly aggregated and organized into a dataset that can directly participate in subsequent time-series analysis. This establishes a stable data input foundation for identifying abnormal tracking behaviors that are repeatedly triggered within a short period. The entire process forms a continuous processing chain from data generation, transmission, access to processing completion.

[0045] S102: Group the log data of each tracking point to be tested according to the release version, business platform, terminal identifier and date to obtain each comparison group, and sort the log data of each tracking point to be tested according to the sending time of the tracking point within each comparison group.

[0046] Among them, "release version" refers to the distinguishable software version identifier formed after an application is released externally or built internally, used to reflect differences in code status and function set; "business platform" refers to the system environment category that supports the application's operation, used to distinguish business forms under different operating systems or operating frameworks; "terminal identifier" refers to the unique identification information that can distinguish different devices or different installation instances, used to characterize the source of logs; "date" is the natural day dimension data extracted based on time information, used to limit the time range of log generation; "group to be compared" refers to the data set unit formed after the field division is completed, used for subsequent repetitive analysis; "tracking point sending time" refers to the time stamp corresponding to when the tracking point initiates the reporting behavior from the terminal, used to reflect the chronological relationship of the behavior.

[0047] In actual business operations, there may be adjustments to the tracking logic between different release versions, and the event triggering mechanisms and interface refresh mechanisms of different business platforms may also differ. Different terminals will generate different log sequences due to differences in user operation paths, and the usage scenarios of the same terminal on different days will also have significant differences. To ensure that subsequent multi-occurrence identification is based on comparable data, we can first read the date information from the release version field, business platform field, terminal identifier field, and time field of the log records. These four types of fields are combined to form grouping keys. Then, based on the consistency of the key values, the logs are divided into multiple data sets, ensuring that the logs in the same set all come from the same version, the same operating environment, the same terminal, and fall within the same natural day range. In this way, the originally mixed full logs are split into several data units with a unified operating background, creating the preconditions for subsequent behavioral pattern analysis within a local scope.

[0048] Within each established dataset, the timing field of each log entry can be further read and rearranged based on the magnitude of the timing values ​​to ensure the log order matches the actual trigger sequence. When dealing with large log volumes, the data within each dataset can be loaded into a memory buffer before a time-based sorting algorithm is used to rearrange the order, placing earlier records at the top and later records at the bottom. After sorting, each dataset forms a continuous time-series structure, making the timing triggering process a traceable behavioral chain at the data level. This process ensures that the position of any log entry within the dataset reflects its temporal relationship with preceding and following actions, laying a time-series foundation for identifying abnormally dense triggers within a short period. The entire process, from field extraction and key-value construction to dataset partitioning and time-series rearrangement, forms a continuous data processing workflow.

[0049] By grouping the log data of the event tracking points to be detected according to the release version, business platform, terminal identifier, and date, the originally mixed full logs are confined to the same code state, the same operating environment, the same device source, and the same time range. This eliminates interference caused by cross-version differences, cross-platform mechanism differences, and differences in user behavior at the data source level, ensuring that the data within the same group has a unified business background and operating conditions, thus establishing a comparable premise for subsequent comparisons. On this basis, the logs within the group are further sorted according to the order of event tracking point sending time, so that the log arrangement order is consistent with the actual occurrence order of behavior, constructing a continuous and traceable time series structure, making the chronological relationship between each record clearly distinguishable, and avoiding judgment bias caused by time misalignment or data disorder. The above processing ensures that each comparison group has both environmental consistency and temporal continuity, enabling subsequent identification of duplicate or abnormal triggers based on temporal proximity relationships to be based on real data sequences under the same conditions, thereby improving the accuracy and stability of the comparison results and providing a reliable data organization foundation for identifying multiple event tracking point issuance behaviors.

[0050] S103: Within each sorted comparison group, the first detection point log data is used as the current starting point of the sliding window, and the preset time length is used as the length of the sliding window. The detection point log data within the sliding window are compared. Based on the comparison result, duplicate detection point log data are recorded, and the current starting point of the sliding window is moved backward by one detection point log data. The comparison continues until the current starting point of the sliding window moves to the last detection point log data within the comparison group.

[0051] Among them, a sliding window refers to a local data interval selected on time series data according to a fixed time span, used to observe log distribution characteristics within a limited time range; the current starting point refers to the starting log position corresponding to the sliding window in the time series, used to determine the starting position of the current comparison interval; the preset time length refers to a pre-set time range value, used to limit the size of the time interval covered by the window, in order to characterize the density of behavior in a short period of time; duplicate event log data refers to log records that are judged to be reported multiple times within the same window time range, with the same semantics and no essential change in the triggering conditions.

[0052] In real-world applications, API calls, element exposures, or user actions on the same page may repeatedly trigger the same event tracking points within a short period. This results in highly concentrated and semantically repetitive log data. To identify these dense triggers, the first log entry in the sorted time-series data can be used as the starting point of a sliding window. A fixed-length window is then set forward from this log entry's time. The data within this window represents the set of candidate logs to be compared within a short period. By traversing each log entry within the window, its event tracking identifier and key trigger-related fields can be obtained to determine whether it has duplicate semantics or triggering behavior with other logs within the window. The determination method can be based on field consistency, trigger condition matching, or other rule settings. Logs identified as duplicates are marked or recorded, thus forming the identification result of duplicate event tracking points within a local time range.

[0053] After completing the log comparison within the current sliding window, the window's starting point is moved forward by one log entry, aligning the new starting point with the next chronologically ordered log entry. A new fixed-length window is then constructed using this new starting point, and the comparison process is repeated. During each sliding operation, the time interval covered by the window advances continuously across the entire time series, allowing each log entry to participate in the comparison of its own window as well as subsequent windows with adjacent times. This line-by-line sliding approach ensures the continuity and integrity of data coverage, preventing the omission of duplicate events due to fixed interval divisions, thus enabling a comprehensive scan of frequently triggered behaviors within a short period.

[0054] For large-scale log data, the grouped logs can be loaded into memory or a buffer and sorted by sending time. An efficient time window algorithm can then be used to construct and compare sliding windows, improving processing efficiency. During comparison, the time difference is calculated to ensure it is within a preset time period, and key field matching and tracking are performed to accurately locate and mark duplicate logs. In this way, short-term repeated triggering behaviors within each comparison group can be identified promptly, forming a continuously covering scanning chain, providing a clear data foundation for subsequent tracking optimization, deduplication, or anomaly handling.

[0055] By using the first log entry as the starting point of a sliding window within each sorted comparison group and defining the window range with a preset time length, and comparing log entries one by one within the window, we can accurately identify multiple triggers of semantically identical event tracking behavior within a short period of time. Furthermore, by shifting the window starting point sequentially, we ensure continuous coverage of the entire time series, guaranteeing that each log entry is compared with its temporally adjacent records, thus avoiding omissions. This implementation captures the characteristics of high-frequency, repetitive triggers over time, ensuring that duplicate event tracking data is completely identified. This provides an accurate basis for subsequent event tracking deduplication, anomaly analysis, or logic optimization, improving the accuracy and stability of detecting multiple event tracking behaviors, while also ensuring the comprehensiveness and traceability of data processing results.

[0056] In the above embodiments, the logs are first grouped according to release version, business platform, terminal identifier, and date. This confines the originally mixed full logs to a comparable range within the same version, environment, terminal, and time dimension, eliminating the interference of cross-version differences, platform differences, and user differences on the judgment results and establishing a unified data benchmark for repetitive determination. Secondly, the logs are sorted by sending time within the group to construct a time sequence of actual behavior, giving the trigger relationship of the event tracking a traceable temporal logic basis. Furthermore, by setting a sliding window of fixed time length, the event tracking logs within a local time interval are compared window by window. This can capture dense reporting behavior caused by repeated interface calls, repeated UI refreshes, or abnormal exposure logic within a short period of time. Compared with the global statistical method, it can more accurately characterize the time-related feature of multiple occurrences. At the same time, the sliding window slides one log at a time according to the log granularity to achieve complete coverage of the entire sequence and ensure the integrity of the detection. Based on this, this application can not only automatically locate duplicate event log data and identify abnormal high-frequency triggering patterns, but also provide data basis for subsequent event deduplication, rule optimization and trigger logic rectification, thereby reducing the amount of invalid data, saving transmission and storage resources, improving the authenticity and usability of event data, and ultimately achieving effective identification and controllable management of the problem of frequent event occurrences.

[0057] In one embodiment, after the step of obtaining the log data of the event tracking point to be detected, the following steps are included:

[0058] Based on a preset blacklist, the log data of each monitoring point to be detected is filtered.

[0059] The preset blacklist refers to a set of predefined tracking point identifiers or specific trigger conditions in the log analysis process. The tracking point data corresponding to these identifiers or conditions are deemed not to need to participate in repetitive analysis or have known abnormal characteristics, and can be directly excluded or ignored.

[0060] In practical applications, some event tracking points may be designed not to participate in normal data analysis due to testing, debugging, or special business logic, but they still generate logs during operation. If these logs are not processed, they will interfere with repetitive detection or lead to false positives. To exclude these logs, a pre-defined blacklist can be loaded into memory, and each log entry in the set to be tested can be read for its tracking point identifier or key trigger condition and compared with the blacklist. If a log entry's tracking point identifier or trigger condition matches in the blacklist, that log entry is removed from the set to be tested or marked to prevent it from participating in subsequent repetitive or abnormal analysis. By using the blacklist as a data filtering standard, invalid or interfering data can be reduced in the early stages of analysis, improving processing efficiency.

[0061] In large-scale log environments, batch comparison or hash indexing can be used for blacklist filtering. A fast query relationship is established between the log set to be detected and the blacklist, and logs in the set are scanned sequentially for matching and filtering, thus achieving efficient processing. After filtering, the remaining log set is a cleaned and effective data set that can be used for duplicate detection, ensuring that subsequent sliding window comparisons or multiple-occurrence detection are based on reliable input data. This process also allows for preliminary control over sensitive data such as log sources and triggering scenarios, preventing abnormal or invalid records from biasing the analysis results.

[0062] By filtering the log data of the event tracking points to be detected based on a preset blacklist, known invalid, debugging, or abnormally triggered log records can be removed before analysis. This ensures that subsequent duplicate detection is only performed on valid data, thereby improving the accuracy and reliability of the analysis results. At the same time, it reduces the amount of processing and improves efficiency. Furthermore, it ensures the consistency and controllability of data cleaning when running in parallel across multiple terminals, versions, and business scenarios, providing a clean, comparable, and reliable data foundation for subsequent multiple event identification.

[0063] In one embodiment, the step of grouping the log data of each monitoring point to be detected according to the release version, business platform, terminal identifier, and date to obtain each comparison group includes:

[0064] Based on a preset terminal identifier whitelist, the log data corresponding to the terminal identifiers in the log data to be detected are filtered out, and then grouped according to the release version, business platform, terminal identifier and date to obtain each comparison group.

[0065] Among them, the preset terminal identifier whitelist refers to a set of unique identifiers for a specific terminal that is defined in advance. These terminals are identified as trustworthy or sample objects for analysis. Their reported event logs are representative and reliable, and can be used as a priority data source for multiple detection.

[0066] In practical applications, different terminals may have differences in usage environment, operating habits, or version status. Some terminal logs may be abnormal or unrepresentative. If the full data is analyzed directly, it may affect the accuracy of multiple detection. To address this, a preset terminal identifier whitelist can be loaded into memory first. Then, the log data of the event tracking points to be detected can be traversed sequentially, and the terminal identifier of each log is compared with the whitelist. Only matching log records are retained. This ensures the credibility of the source of the analysis object and reduces interference from data from non-target terminals. After filtering, the remaining logs can be combined according to the values ​​extracted from the release version, business platform, terminal identifier, and date fields. Data with the same version, platform, terminal, and date can be grouped into the same data set, giving the data within each set a unified business background and time dimension, facilitating subsequent behavior comparison.

[0067] In large-scale log scenarios, indexing or hash mapping can be used to achieve fast whitelist matching, ensuring an efficient and stable filtering process. After filtering, logs within each set are sorted by sending time to ensure the log order matches the actual trigger sequence, thus forming comparison groups that possess both background consistency and temporal continuity. These groups can directly participate in sliding window comparisons or other repetitive analyses, ensuring that multiple occurrence detection is based on a reliable, comparable, and ordered data foundation.

[0068] By first filtering logs based on a pre-defined whitelist of terminal identifiers, and then grouping them according to release version, business platform, terminal identifier, and date, it is possible to ensure that the source of the analysis objects is reliable and the background conditions are consistent, forming a structured data set that can be directly used for duplicate identification. This improves the accuracy and stability of duplicate detection, reduces interference from invalid data, and ensures that subsequent comparison and analysis can be carried out smoothly based on a reliable and traceable data sequence when running in parallel with multiple terminals, multiple versions, and multiple business scenarios, providing a reliable data foundation for the identification and optimization of event tracking anomalies.

[0069] In one embodiment, the step of comparing the log data of the event tracking points to be detected within the sliding window and recording duplicate event tracking point log data based on the comparison result includes:

[0070] Based on the key business field content in each log data point to be detected within the sliding window, determine the summary identifier of each log data point to be detected.

[0071] Each log data point to be detected within the sliding window is compared pairwise according to its corresponding summary identifier. Log data points with the same summary identifier are recorded as duplicate log data points.

[0072] Among them, the key business field content refers to the log fields that can characterize the semantics and context of the event tracking, including event type, operation object, interface location or function identifier, etc. These fields can reflect the essential characteristics of the event tracking behavior.

[0073] In real-world applications, the same page operation, API callback, or exposure behavior may generate multiple semantically identical event logs within a very short period. To accurately identify these duplicate behaviors, we can first read the key business fields of each log in the sliding window and calculate a summary identifier based on the field values. The summary identifier can be generated using hash algorithms, field concatenation, or rule encoding, ensuring that logs with the same semantic event log have the same identifier, while logs with different semantics or trigger conditions generate different identifiers. After generating the summary identifier, the logs in the sliding window are compared pairwise according to their summary identifiers. By matching the summary identifiers, we can determine which logs are duplicate triggers. The matching logs are recorded as duplicate event log data and stored or marked for subsequent processing.

[0074] In large-scale sliding window environments, digest identifiers can be mapped to an index structure, such as a hash table or dictionary, to group logs with the same identifier into the same set. Duplicate records are then marked within each set. This significantly improves comparison efficiency and avoids brute-force pairwise comparisons between each log entry and all window logs. During execution, each log entry generates a unique digest based on its key business fields, ensuring that duplicate identification relies strictly on semantic features rather than time or other auxiliary fields. This guarantees that frequently triggered logs within a short period can be accurately captured and marked, forming a clear and traceable set of duplicate records.

[0075] By generating summary identifiers based on the key business fields of each log entry within a sliding window and comparing these summary identifiers, semantically identical and different event tracking logs can be accurately distinguished. This ensures that logs triggered repeatedly within a short period are accurately identified as duplicate event tracking data, while preventing non-duplicate logs from being misjudged. This provides a reliable basis for subsequent deduplication, anomaly analysis, or event tracking logic optimization. This method improves processing efficiency and controllability while maintaining identification accuracy, making duplicate detection results stable and traceable, and providing a structured and accurate data foundation for multi-issue behavior analysis.

[0076] In one embodiment, the step of determining the summary identifier of each log data point to be detected based on the content of key business fields in each log data point to be detected within the sliding window includes:

[0077] After replacing the dynamic variables that conform to the preset rules in the key business fields of each log data point to be detected within the sliding window with static unified values, a summary identifier for each log data point to be detected is generated using a preset summary algorithm.

[0078] Among them, dynamic variables refer to the content in the log field that changes with each trigger, such as user ID, session ID, timestamp, or randomly generated values; static unified values ​​refer to the fixed values ​​after replacing dynamic variables, used to eliminate irrelevant differences in different logs and ensure that the same semantic tracking points generate consistent identifiers; preset rules refer to the set of rules that determine which fields are dynamic variables and how to replace them with static unified values; and the digest algorithm refers to generating unique or nearly unique identifiers by performing hash calculations, encoding, or other algorithmic processing on the log field content, used to quickly distinguish different logs or identify duplicate logs.

[0079] In practical applications, identical operations within a short period may contain different dynamic variables in logs, such as user IDs, timestamps, or random numbers, making direct comparison difficult even if the original logs are semantically identical. To achieve semantic consistency identification, we can first read the key business fields of each log entry within a sliding window and identify the dynamic variables according to preset rules, replacing them with static, unified values. This ensures that logs with different triggers but the same semantics maintain consistency in field representation. After replacement, the processed field content can be input into a preset digest algorithm to generate a unique digest identifier through hash functions, encryption encoding, or other rules. This ensures that logs with the same semantics generate the same identifier, while logs with different semantics or trigger conditions generate different identifiers.

[0080] In a large-scale log environment, the log field content within the sliding window can be loaded into a memory buffer first. A rule parser then identifies and replaces dynamic variables one by one. The replaced fields are then combined and input into an efficient digest algorithm to calculate an identifier. The calculation result can be stored in an index structure for rapid comparison. In this way, the digest identifier generated for each log entry strictly depends on its business semantics rather than random or dynamically changing fields, ensuring the accuracy of duplicate identification. It also provides a unified identifier basis for subsequent sliding window comparisons, enabling logs with the same semantics to be accurately identified as duplicate data points when triggered intensively over time.

[0081] By replacing dynamic variables in the logs within a sliding window with static, uniform values ​​and generating summary identifiers, irrelevant dynamic differences in the logs can be eliminated, ensuring that logs with the same semantics receive the same identifier, thus achieving high-precision identification of duplicate event tracking. This method ensures that logs triggered intensively in a short period are accurately identified as duplicate event tracking, while avoiding misjudgments caused by dynamic variables. It improves the accuracy and stability of duplicate identification, providing a reliable and traceable data foundation for subsequent deduplication, anomaly analysis, and optimization of event tracking logic.

[0082] In one embodiment, the step of moving the current starting point of the sliding window backward by one log data point to be detected and continuing the comparison includes:

[0083] The next event log data to be detected at the current starting point of the sliding window is taken as the target event log data, and it is determined whether the target event log data has been recorded as duplicate event log data.

[0084] If not, the target event log data will be used as the new starting point of the sliding window, and the comparison will continue.

[0085] If so, the target event log data will no longer be used as the current starting point of the sliding window, and the next event log data to be detected will be used as the new target event log data to continue to determine whether it has been recorded as duplicate event log data.

[0086] Among them, the target tracking log data refers to the next log record to be detected after the current starting point of the sliding window, which is used to determine whether it belongs to repeated triggering behavior.

[0087] In practical applications, after the sliding window completes the duplicate comparison of the current starting log, it is necessary to determine whether the next log can be used as a new comparison starting point to ensure continuous scanning of the entire time series and complete identification of duplicate triggering behavior. The next log from the current starting point can be set as the target event log data, and it is first determined whether this log has already been marked as duplicate event log data. If the target log has not yet been marked as duplicate, it is used as the new starting point of the sliding window, and duplicate comparison continues within the new time window, thus ensuring that every unmarked log has a chance to participate in duplicate identification. If the target log has already been marked as duplicate, this log is skipped and no longer used as the starting point of the sliding window. Instead, the next log is directly set as the new target log for judgment, and so on, ensuring that marked logs are not repeatedly compared, thus avoiding redundant calculations and resource waste. In a large-scale sliding window environment, marked duplicate logs can be stored in a fast-retrieval data structure, such as a hash table or indexed array, to achieve efficient comparison for target log judgment. As the time series is scanned one by one and the starting point of the sliding window is dynamically moved according to the above rules, each log can selectively participate in the comparison based on its marking status, forming a complete and thorough duplicate identification link, while ensuring processing efficiency and identification accuracy.

[0088] For example, in an e-commerce app, when a user quickly refreshes the product list, the same product's exposure event might trigger three times within one second. Key business fields include product ID, page position, and exposure event type. During sliding window processing, the first exposure log can be used as the starting point. A summary identifier is calculated and compared with other logs within the window to identify and mark duplicate exposure logs as duplicate events. After completing the window comparison, the next log is set as the target log. If the log is not marked as duplicate, it is set as the new starting point for the sliding window comparison, ensuring that all unmarked logs within the time series are included in the detection. If the target log is already marked as duplicate, it is skipped, and its next log is used as the new target log for judgment, thus avoiding duplicate processing of the same exposure log. In large-scale log scenarios, marked duplicate logs can be stored in a hash table. Even if hundreds of identical product exposure logs are generated in a short period of time, each log can still be efficiently judged to be duplicated, ensuring the integrity and accuracy of duplicate identification. At the same time, it reduces the consumption of computing resources and forms a complete closed loop from time series reading, target log judgment, sliding window start-up update to the next round of comparison, providing a reliable and traceable data foundation for subsequent data collection, deduplication, anomaly analysis and optimization.

[0089] By setting the next log entry to be detected from the current starting point of the sliding window as the target log, and first determining whether it has been marked as a duplicate log entry, it is ensured that every unmarked log has a chance to participate in the duplicate comparison, while avoiding duplicate comparisons of marked logs. When the target log is not marked as duplicate, it is used as the new starting point of the sliding window for continued comparison, ensuring that duplicate triggering behavior of consecutive logs within the time series can be fully identified; when the target log has been marked as duplicate, it is skipped directly, and the next log is used as the new target log for judgment, thereby avoiding duplicate processing of already identified logs. This implementation method can improve the efficiency of duplicate comparison, reduce computational resource consumption, and ensure the completeness and accuracy of duplicate log entry identification while continuously scanning the entire time series, so that duplicate logs are accurately marked and not missed, providing a reliable and traceable data foundation for subsequent data deduplication, anomaly analysis, and log entry optimization.

[0090] In one embodiment, the method further includes:

[0091] For two groups to be compared on adjacent natural days, obtain the log data of the monitoring point to be detected in the previous group to be compared whose sending time is later than the first threshold time point, and the log data of the monitoring point to be detected in the next group to be compared whose sending time is earlier than the second threshold time point, to form a cross-day detection set.

[0092] Within the cross-day detection set, the log data of each detection point is sorted according to the sending time of the detection point, and duplicate detection point log data is identified and recorded.

[0093] Here, adjacent natural days refer to two consecutive days in calendar order, used for cross-day continuity analysis; the cross-day detection set refers to the log set formed by merging log data sent later than the first threshold time point in the first group and log data sent earlier than the second threshold time point in the second group from two comparison groups of adjacent natural days, used for cross-day duplication detection; the first threshold time point and the second threshold time point refer to preset times used to define the time range before and after the cross-day boundary;

[0094] In practical applications, cross-day operations may result in duplicate logs generated by continuous behavior across different date groups. For example, a user might refresh a product list at 23:59 and then refresh the same list again at 00:01 the next day, causing exposure tracking points to be triggered consecutively across different day groups. Without cross-day processing, these duplicate logs might be missed. A better approach is to first acquire log data from the previous day group whose sending time is later than a first threshold time point, and log data from the subsequent day group whose sending time is earlier than a second threshold time point, merging them into a cross-day detection set. Then, the logs in the cross-day detection set are sorted according to the timing of the tracking points, and summary identifiers or key field comparisons are performed sequentially to identify logs that are repeatedly triggered within a short period at the cross-day boundary, and these are recorded as duplicate tracking point log data.

[0095] In large-scale application scenarios, logs from the cross-day detection set can be loaded into high-speed storage or a memory buffer and quickly sorted according to time order. Then, comparisons are made based on summary identifiers or key business fields to ensure accurate capture of consecutive cross-day triggering behaviors. For example, in an e-commerce app, if a user refreshes the product list at 23:59, triggering an exposure event for product A, and then refreshes the same product list again at 00:01 the next day, triggering another exposure event, these two logs will be correctly identified as duplicate events through cross-day detection set comparison, even though they belong to different day groups. This approach ensures that the identification of continuous, repetitive behaviors across days is not interrupted, while guaranteeing the accuracy and completeness of duplicate event data labeling, providing a reliable and traceable data foundation for subsequent deduplication, anomaly analysis, and event optimization.

[0096] By aggregating cross-day logs from adjacent natural days and comparing them in chronological order of transmission, we can accurately identify repeatedly triggered event logs within a short period at the cross-day boundary, avoiding the omission of duplicate behavior due to natural day grouping. This implementation not only ensures the completeness and accuracy of cross-day duplicate detection but also improves processing efficiency, making the duplicate identification results stable and controllable, and providing a reliable foundation for subsequent data deduplication, anomaly analysis, and event log optimization.

[0097] To facilitate understanding of the scheme in this application, specific examples are provided below.

[0098] During app development, there is a common occurrence of multiple tracking points. Typical scenarios include sending requests to multiple APIs from the same page, asynchronously refreshing a UI element, and during the exposure of a product list, repeated reporting of tracking points for a particular product slot due to swiping operations. These duplicate tracking points not only waste storage and bandwidth resources but also severely impact the accuracy of subsequent BI data analysis.

[0099] To address this issue, event tracking points generated during routine internal testing are extracted and stored in a database. The tracking data is grouped by App release version, business platform, mobile phone midpoint, and date. Within each group, the tracking data is arranged chronologically by sending time. By creating a sliding window within each group, log data in the queue can be compared sequentially within a set time range, specifically the MD5 hashes of key JSON fields such as `activity_param` and `activity_data`. If the MD5 hashes match, these tracking points are considered duplicates and recorded in the database. The sliding window moves sequentially from first to last, discarding tracking points that exceed the time threshold, thus improving comparison efficiency and avoiding unnecessary calculations.

[0100] In practical applications, the event logs collected by the test agent contain trigger context information, such as the event trigger type (click, impression, and page lifecycle callbacks) and business-related fields (order number and order status). This context information is highly relevant to the business, and reasonable duplication is generally not present for event logs that are of business interest. For example, for event logs related to order list impressions, different order numbers correspond to different impressions; if the same order number appears, it is considered a duplicate. General technical event logs, such as switching from the backend to the foreground or black screen events, can be filtered through a blacklist to avoid interfering with business event log analysis.

[0101] In the grouping dimension, if the test environment involves emulators reusing mobile phone midpoints in batches, it may cause the event tracking points of different test devices to be classified into the same group, thus affecting the duplicate detection results. To address this, the system uses an internal test whitelist, focusing only on the midpoints of real test users. Additionally, for short-term duplicate event tracking points spanning multiple dates, an overlap check is added at the date threshold to avoid missing duplicate event tracking points across dates due to date-based grouping.

[0102] The time threshold settings have also been validated across different business scenarios. For high-frequency operation scenarios, such as when a user clicks a button repeatedly within one second, the app has a mechanism to prevent duplicate clicks, ensuring that event tracking is not sent repeatedly within a short period. Dynamic fields in the JSON, such as timestamps, are uniformly replaced by the system to ensure that actual duplicate event tracking can be accurately identified and to avoid missed detections due to different hash values ​​caused by dynamic fields.

[0103] The sliding window moves stepwise, checking each tracking point individually. Starting with the first tracking point in the time series, it checks all tracking points within a three-second time range. After completing the check of the current window, the window's starting point moves to the next tracking point to continue the comparison. While this method may involve duplicate calculations, it ensures that no duplicate tracking points are missed. During the sliding process, tracking points at the end of the previous window and the beginning of the next window are effectively identified if the interval is within three seconds, eliminating the issue of missed detections across windows. To further improve efficiency, tracking point code groups can be added to the phone number and date groupings, making the tracking points within the three-second window sparser, thereby reducing unnecessary comparisons while maintaining accuracy.

[0104] This method has been proven in practice to accurately identify more than thirty types of duplicate tracking points and automatically performs checks via scheduled tasks, requiring no manual intervention or additional code maintenance. This systematic approach not only improves the accuracy of duplicate tracking point identification but also optimizes the use of computing resources, providing a reliable data foundation for subsequent tracking point data analysis and business decision-making.

[0105] The following describes the multiple-occurrence detection device for event tracking provided in the embodiments of this application. The multiple-occurrence detection device described below can be referred to in correspondence with the multiple-occurrence detection method described above. Figure 2 As shown, this application provides a device for detecting multiple embedded points, the device comprising:

[0106] The module 201 for acquiring log data of the tracking points to be detected is used to acquire log data of the tracking points to be detected.

[0107] The comparison group determination module 202 is used to group the log data of each monitoring point to be tested according to the release version, business platform, terminal identifier and date to obtain each comparison group, and sort the log data of each monitoring point to be tested according to the sending time of the monitoring point within each comparison group.

[0108] The duplicate event log data recording module 203 is used to compare the event log data to be detected within each sorted comparison group, taking the first event log data to be detected as the current starting point of the sliding window and the preset time length as the length of the sliding window. Based on the comparison result, the duplicate event log data is recorded, and the current starting point of the sliding window is moved backward by one event log data to be detected. The comparison continues until the current starting point of the sliding window moves to the last event log data to be detected within the comparison group.

[0109] In one embodiment, after the module 201 for acquiring the log data of the event tracking point to be detected, the following is included:

[0110] The module for filtering log data of the event tracking points to be detected is used to filter the log data of each event tracking point to be detected based on a preset blacklist.

[0111] In one embodiment, the comparison group determination module 202 includes:

[0112] The comparison group determination unit is used to filter the log data corresponding to the terminal identifiers in the whitelist of the terminal identifiers to be detected in the log data of the monitoring points based on the preset terminal identifier whitelist, and then group them according to the release version, business platform, terminal identifier and date to obtain each comparison group.

[0113] In one embodiment, the repetitive data recording module 203 includes:

[0114] The summary identifier determination unit is used to determine the summary identifier of each log data point to be detected based on the content of key business fields in each log data point to be detected within the sliding window.

[0115] The duplicate event log data recording unit is used to compare each event log data to be detected in the sliding window with the corresponding summary identifier, and record the event log data with the same summary identifier as duplicate event log data.

[0116] In one embodiment, the digest identifier determination unit includes:

[0117] The summary identifier determination sub-unit is used to replace dynamic variables that conform to preset rules in the key business fields of each log data point to be detected within the sliding window with static unified values, and then generate a summary identifier for each log data point to be detected through a preset summary algorithm.

[0118] In one embodiment, the repetitive data recording module 203 includes:

[0119] The target event log data determination unit is used to take the next event log data to be detected from the current starting point of the sliding window as the target event log data, and to determine whether the target event log data has been recorded as duplicate event log data.

[0120] The sliding window start point determination unit is used to determine the target tracking point log data as the new current start point of the sliding window and continue the comparison if the target tracking point log data is not found.

[0121] The target event log data update unit is used to, if yes, no longer use the target event log data as the current starting point of the sliding window, and use the next event log data to be detected as the new target event log data, so as to continue to determine whether it has been recorded as duplicate event log data.

[0122] In one embodiment, the apparatus further includes:

[0123] The cross-day detection set construction module is used to obtain the detection point log data in the previous comparison group whose sending time is later than the first threshold time point, and the detection point log data in the next comparison group whose sending time is earlier than the second threshold time point, for two comparison groups on adjacent natural days, to form a cross-day detection set.

[0124] The cross-day detection set identification module is used to sort the log data of each detection point in the cross-day detection set according to the sending time of the detection point, and then identify and record the duplicate detection point log data.

[0125] In one embodiment, this application also provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the multiple-time detection method for embedded points as described in any of the above embodiments.

[0126] In one embodiment, this application also provides a computer device storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the multiple-time detection method for embedded points as described in any of the above embodiments.

[0127] Indicatively, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the internal structure of a computer device 300 provided in an embodiment of this application. The computer device 300 can be provided as a server. (Refer to...) Figure 3 The computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions, such as application programs, that can be executed by the processing component 302. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 302 is configured to execute instructions to perform the multi-point detection method of any of the above embodiments.

[0128] The computer device 300 may also include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305. The computer device 300 may operate on an operating system stored in memory 301, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0129] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0130] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this document, "a," "an," "the," "the," and "its" may also include plural forms unless the context clearly indicates otherwise. "Multiple" refers to at least two, such as 2, 3, 5, or 8, etc. "And / or" includes any and all combinations of the related listed items.

[0131] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0132] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting a multi-hit burying point, characterized in that, The method includes: Obtain the log data of the event tracking points to be detected; The log data of each of the event tracking points to be detected are grouped according to the release version, business platform, terminal identifier and date to obtain each comparison group. Within each comparison group, the log data of each event tracking point to be detected are sorted according to the order of the event tracking point sending time. Within each sorted comparison group, the first detection log data is used as the current starting point of the sliding window, and a preset time length is used as the length of the sliding window. The detection log data within the sliding window are compared. Based on the comparison result, duplicate log data are recorded, and the current starting point of the sliding window is moved backward by one detection log data. The comparison continues until the current starting point of the sliding window moves to the last detection log data within the comparison group.

2. The method of claim 1, wherein, After the step of obtaining the log data of the event tracking point to be detected, the following steps are included: Based on a preset blacklist, the log data of each of the aforementioned monitoring points is filtered.

3. The method of claim 1, wherein, The step of grouping the log data of each monitoring point to be detected according to the release version, business platform, terminal identifier, and date to obtain each comparison group includes: Based on a preset terminal identifier whitelist, log data corresponding to terminal identifiers belonging to the whitelist are obtained from the log data of the event tracking points to be detected. Then, the data is grouped according to the release version, the business platform, the terminal identifier, and the date to obtain each comparison group.

4. The method of claim 1, wherein, The step of comparing the log data of the event tracking points to be detected within the sliding window and recording the duplicate event tracking point log data based on the comparison result includes: Based on the key business field content in each of the log data to be detected within the sliding window, determine the summary identifier of each log data to be detected. Each of the log data points to be detected within the sliding window is compared pairwise according to its corresponding summary identifier. Log data points to be detected with the same summary identifier are recorded as duplicate log data points.

5. The method of claim 4, wherein, The step of determining the summary identifier of each piece of log data to be detected based on the key business field content in each piece of log data to be detected within the sliding window includes: After replacing the dynamic variables that conform to preset rules in the key business fields of each of the log data to be detected within the sliding window with static unified values, a summary identifier for each of the log data to be detected is generated using a preset summary algorithm.

6. The method of claim 1, wherein, The step of moving the current starting point of the sliding window backward by one log data point to be detected and continuing the comparison includes: The next undetected event log data at the current starting point of the sliding window is taken as the target event log data, and it is determined whether the target event log data has been recorded as duplicate event log data. If not, the target data point log is used as the new starting point of the sliding window, and the comparison continues. If so, the target event log data will no longer be used as the current starting point of the sliding window, and the next event log data to be detected will be used as the new target event log data to continue to determine whether it has been recorded as duplicate event log data.

7. The method of claim 1 to 6, wherein, The method further includes: For two comparison groups on adjacent natural days, the detection log data of the tracking point in the preceding comparison group whose sending time is later than the first threshold time point is obtained, and the detection log data of the tracking point in the following comparison group whose sending time is earlier than the second threshold time point is obtained, thus forming a cross-day detection set. Within the cross-day detection set, the log data of each detection point is sorted according to the sending time of the detection point, and duplicate detection point log data is identified and recorded.

8. A device for detecting multiple hits of a buried point, characterized in that The device includes: The module for acquiring log data of the event tracking points to be detected is used to acquire log data of the event tracking points to be detected. The comparison group determination module is used to group the log data of each of the monitoring points according to the release version, business platform, terminal identifier and date to obtain each comparison group, and sort the log data of each monitoring point according to the sending time of the monitoring point within each comparison group. The duplicate event tracking log data recording module is used to compare the event tracking log data within each sorted comparison group, using the first event tracking log data to be detected as the current starting point of the sliding window and a preset time length as the length of the sliding window. Based on the comparison result, duplicate event tracking log data is recorded, and the current starting point of the sliding window is moved backward by one event tracking log data to be detected. The comparison continues until the current starting point of the sliding window moves to the last event tracking log data to be detected in the comparison group.

9. A storage medium characterized by: The storage medium stores computer-readable instructions, which, when executed by one or more processors, cause the one or more processors to perform the steps of the multiple-time detection method for embedded points as described in any one of claims 1 to 7.

10. A computer device, comprising: include: One or more processors, and memory; The memory stores computer-readable instructions, which, when executed by the one or more processors, perform the steps of the multiple-time detection method for embedded points as described in any one of claims 1 to 7.

Images