Method, device, equipment and medium for filtering and processing duplicate buried point data

Abstract

The present invention discloses a method, device, equipment and medium for filtering and processing repeated buried point data. Its features include: filtering and matching the buried point data based on the data filtering array sent by the server to determine at least one target matching data; obtaining the data reporting method of each target matching data, and querying the number of buried point data of each target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method; processing the target matching data according to the number of buried point data of each target matching data, and determining the data processing result. It can filter a large number of repeated and invalid buried point data, and flexibly process the repeated buried point data based on different reporting methods of buried point data, thereby improving the stability of the program and the user experience.

Description

Technical Field

[0001] The present invention relates to the field of software management technology, and in particular to a filtering and processing method, device, equipment and medium for repeated buried point data. Background Art

[0002] The full-link tracking SDK (Software Development Kit) is a toolkit for collecting user behavior data of the Web (World Wide Web) and mobile APP (Application). The full-link tracking SDK can collect data at each key node of the full link, and achieve comprehensive coverage of the entire business process from the front end to the back end; after the tracking data is completed, it supports two ways to report the tracking data, one is internal data reporting, and the other is third-party reporting. At present, the user behavior between the car machine and the mobile APP is unpredictable, which makes the mobile APP unable to control the actual data collection time and tracking number, resulting in many tracking points being repeated and invalid within a certain period of time, making analysis difficult. If a large number of tracking points are reported to the server, the server data will increase sharply in a short period of time. Invalid tracking points will also cause the server memory to crash. In addition, when checking the filtering strategy, the input data needs to be stored in the mobile APP memory for the next check. If it is not cleaned up in time, the memory check data on the mobile APP will only increase, which will also cause the mobile APP memory to crash. It is difficult to collect effective tracking data to analyze user behavior. Summary of the invention

[0003] The present invention provides a filtering and processing method, device, equipment and medium for repeated buried point data, so as to solve the technical problem that the buried point data collected in the prior art are repeated and invalid.

[0004] According to one aspect of the present invention, a method for filtering and processing duplicate buried point data is provided, comprising:

[0005] Filter and match the buried data based on the data filter array sent by the server to determine at least one target matching data; wherein the buried data includes an event identifier, an event result, and an event timestamp;

[0006] Obtaining a data reporting method for each of the target matching data, querying the number of data items of each of the target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determining the number of embedded point data items corresponding to each of the target matching data;

[0007] The target matching data is processed according to the number of embedded data items of each target matching data to determine the data processing result.

[0008] According to another aspect of the present invention, a device for filtering and processing duplicate buried point data is provided, comprising:

[0009] A filtering module, used to filter and match the buried data based on the data filtering array sent by the server, and determine at least one target matching data; wherein the buried data includes an event identifier, an event result and an event timestamp;

[0010] A query module, used to obtain a data reporting method for each of the target matching data, query the number of data items of each of the target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determine the number of embedded point data items corresponding to each of the target matching data;

[0011] The processing module is used to process the target matching data according to the number of buried data of each target matching data and determine the data processing result.

[0012] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the filtering and processing method for duplicate buried point data described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the filtering and processing method for duplicate buried point data described in any embodiment of the present invention when executed.

[0017] The technical solution of the embodiment of the present invention is to filter and match the buried data based on the data filtering array sent by the server, determine at least one target matching data, obtain the data reporting method of each target matching data, query the number of data items of each target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determine the number of buried data items corresponding to each target matching data; perform data processing on the target matching data according to the number of buried data items of each target matching data, determine the data processing result, determine the data to be filtered through data filtering, and determine the corresponding data filtering method based on different data reporting methods, which can effectively support the operation of the project, and then perform data filtering on the data to be filtered, and can filter a large number of repeated and invalid buried data, so as to better monitor and analyze user behavior; solve the technical problem that the buried data collected in the prior art is repeated and invalid. It can effectively enhance the stability of the project and improve the user experience.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flowchart of a method for filtering and processing duplicate buried point data is provided for an embodiment of the present invention;

[0021] Figure 2 A flowchart of another method for filtering and processing duplicate buried point data provided by an embodiment of the present invention;

[0022] Figure 3 A flowchart of another method for filtering and processing duplicate buried point data provided by an embodiment of the present invention;

[0023] Figure 4 A schematic diagram of the structure of a device for filtering and processing duplicate buried point data provided by an embodiment of the present invention;

[0024] Figure 5 A schematic structural diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Figure 1 A flowchart of a method for filtering and processing duplicate buried point data is provided for an embodiment of the present invention. This embodiment can be applied to the case where a large amount of duplicate buried point data is generated by collecting software behaviors through buried points and filtering and processing the duplicate buried point data. The method can be executed by a filtering and processing device for duplicate buried point data. The filtering and processing device for duplicate buried point data can be implemented in the form of hardware or software, and the filtering and processing device for duplicate buried point data can be configured in an electronic device. Figure 1 As shown, the method includes:

[0028] S110: Filter and match the embedded point data based on the data filter array sent by the server to determine at least one target matching data.

[0029] Among them, the buried data can be the user's behavior data or business data on the mobile APP side; it should be noted that the buried data set on the mobile APP side, and then collect the user's behavior data or business data on the mobile APP side to obtain the buried data. The buried data includes event identification, event results and event timestamps; the event identification can be the identification information of the user behavior data, and the event identification information can be used to distinguish different types of user behaviors or business events. The event result can be the final state that reflects the user behavior or business event, which can be used to reflect the business effect or user experience. Exemplarily, the event result can be success, failure and cancellation. For example, for a payment event, if the result is success, it means that the user has completed the payment operation; if the result is failure, it means that the user has not completed the payment operation; the event timestamp records the specific time when the event occurred.

[0030] Among them, the server can be a project server. The server can pre-configure a data filter array for filtering the buried data. The server configures the data filter array and sends down the data filter array. The buried data can be filtered through the data filter array, and the buried data that meets the data filter array can be filtered.

[0031] The data filter array may be used to filter and match the event identifiers and event results of the buried data. Exemplarily, the data filter array may be a filter array.

[0032] Among them, the target matching data can be the buried data obtained by filtering and matching the data filtering array; it should be noted that the data filtering array can filter the buried data corresponding to multiple different event identifiers and event results, and then obtain at least one target matching data.

[0033] Specifically, the server pre-configures a data filter array, sends the configured data filter array, and filters and matches the collected buried data based on the data filter array sent by the server to determine at least one target matching data.

[0034] S120, obtaining a data reporting method for each target matching data, querying the number of data items of each target matching data within a filtering time period corresponding to a data filtering array based on the data reporting method, and determining the number of embedded point data items corresponding to each target matching data.

[0035] Among them, the data reporting method can be a method for reporting tracking data, and the data reporting method includes a first reporting method and a second reporting method. The first reporting method can be a third-party tracking reporting method. The third-party tracking reporting method can assemble the tracking data and pass the data to a third-party SDK for reporting. For example: reporting the tracking data to Tencent Cloud or Alibaba Cloud, etc. The second reporting method can be a self-developed tracking SDK reporting mechanism within the project.

[0036] Among them, the filtering time period can be a time period used to filter the buried point data. The filtering time period can be used to focus on a specific time period by setting a specific time range, that is, a time period constructed within a specific time range from the current timestamp forward.

[0037] The number of buried data items may be the number of target matching data items within the filtering time period. The number of buried data items may be used to indicate the number of times buried data items with the same meaning are generated within the filtering time period.

[0038] Optionally, in an embodiment of the present invention, different data reporting methods have different data processing methods for different buried point data. By querying the data reporting method of each target matching data, the number of buried point data of each target matching data within the filtering time period corresponding to the data filtering array is queried based on the data reporting method.

[0039] Specifically, each target matching data has a corresponding data reporting method. For any target matching data, the data reporting method of the target matching data is obtained, and the number of data items of the target matching data within the filtering time period corresponding to the data filtering array is queried based on the data reporting method, and the number of buried point data items corresponding to the target matching data is determined, thereby querying the number of buried point data items corresponding to each target matching data.

[0040] Optionally, in another optional embodiment of the present invention, filtering and matching the buried point data based on the data filtering array sent by the server to determine at least one target matching data includes:

[0041] The event identifier and event result of the buried data are filtered and matched based on the filtering rules of the data filtering array sent by the server, and each buried data whose event identifier and event result match the filtering rules of the data filtering array is determined as a target matching data.

[0042] Optionally, the server pre-sets filtering rules for filtering and matching embedded point data in the data filtering array, configures the embedded point data field of the filtering rules, sets the event identifier and event result of the embedded point data to be matched, and filters and matches the embedded point data through the event identifier and event result of the filtering rules.

[0043] Optionally, when filtering and matching through the filtering rules of the data filtering array, the event identifier and event result of each buried point data are filtered and matched one by one in turn through the event identifier and event result of the filtering rules. If any event identifier and event result in the filtering rules are the same as the event identifier and event result of the buried point data, the buried point data is considered to be matched successfully, and the successfully matched buried point data is used as a target matching data. If the event identifier and event result of a buried point data are different from any event identifier and event result in the filtering rules, the buried point data is filtered out.

[0044] S130, performing data processing on the target matching data according to the number of buried point data of each target matching data, and determining the data processing result.

[0045] Among them, the data processing result can be the result of data processing on the target matching data; it should be noted that the data processing result of the target matching data can be to filter out the target matching data and not store and report the target matching data; it can also be to store the target matching data and report the target matching data through the data reporting method corresponding to the target matching data.

[0046] Specifically, for each target matching data, data processing is performed on the target matching data according to the number of embedded point data of the target matching data to obtain a data processing result corresponding to each target matching data.

[0047] The technical solution of the embodiment of the present invention is to filter and match the buried data based on the data filtering array sent by the server, determine at least one target matching data, obtain the data reporting method of each target matching data, query the number of data items of each target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determine the number of buried data items corresponding to each target matching data; perform data processing on the target matching data according to the number of buried data items of each target matching data, determine the data processing result, determine the data to be filtered through data filtering, and determine the corresponding data filtering method based on different data reporting methods, which can effectively support the operation of the project, and then perform data filtering on the data to be filtered, and can filter a large number of repeated and invalid buried data, so as to better monitor and analyze user behavior; solve the technical problem that the buried data collected in the prior art is repeated and invalid. It can effectively enhance the stability of the project and improve the user experience.

[0048] Figure 2 A flowchart of another method for filtering and processing duplicate buried data provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is the processing method of the target matching data when the data reporting method corresponding to the target matching data is the first reporting method. Figure 2As shown, the method includes:

[0049] S210: Filter and match the embedded point data based on the data filter array sent by the server to determine at least one target matching data.

[0050] S220. When the data reporting method of the target matching data is the first reporting method, the number of data items of the target matching data within the filtering time period corresponding to the data filtering array is queried in the running memory to determine the number of embedded point data items corresponding to each target matching data.

[0051] The running memory may be a random access memory for storing running apps and data. It should be noted that when each mobile app is running, it will load data and instructions into the running memory, and then access and process the data.

[0052] Optionally, in an embodiment of the present invention, when the data reporting method corresponding to the target matching data is the first reporting method, that is, the buried data needs to be reported through a third party, the collected buried data are all in the running memory, and then the buried data can be processed in the running memory.

[0053] Specifically, for the target matching data, when the data reporting method of the target matching data is the first reporting method, the number of data items of each target matching data within the filtering time period corresponding to the data filtering array is queried in the running memory to obtain the number of buried point data items corresponding to each target matching data.

[0054] Optionally, in another optional embodiment of the present invention, querying the number of embedded point data items of the target matching data within the filtering time period corresponding to the data filtering array in the running memory includes:

[0055] The buried point data in the running memory with the same event identifier and event result as the target matching data is stored in the preset target array;

[0056] The target array is traversed. If the event timestamp of a target matching data in the target array is less than the difference between the current timestamp and the preset time period, the target matching data is determined as the target buried point data, and the number of buried point data corresponding to the target buried point data is recorded.

[0057] Among them, the preset target array can be an array pre-set for storing target matching data; it should be noted that the preset target array can store multiple different target matching data, and for target matching data corresponding to an event identifier and event result, the preset target array can store multiple target matching data with the same event identifier and event result.

[0058] Optionally, when filtering and identifying the buried data through target matching data, the buried data of the event identifier and event result in the running memory and the event identifier and event result of any target matching data are stored in a preset target array as a target matching data stored in the target array.

[0059] The current timestamp may be the real-world timestamp when the target matching data is processed. It should be noted that the current timestamp is the timestamp corresponding to the current time zone in the real world, and as time changes, the time corresponding to the current timestamp also changes.

[0060] The preset time period may be a time value preset for determining the filtering time period. For example, the preset time period may be set to 60 seconds, and the filtering time period may be the current timestamp minus 60 seconds.

[0061] Among them, the target buried data can be the buried data that is valid for the target matching data within the filtering time period corresponding to the data filtering array; optionally, the number of buried data items corresponding to the target buried data is the number of data items within the filtering time period corresponding to the statistical data filtering array, and then in the target array, for any target matching data, the event timestamp of the target matching data is compared with the difference between the current timestamp and the preset time period. If the event timestamp of the target matching data is less than the difference between the current timestamp and the preset time period, it is considered that the target matching data is within the filtering time period corresponding to the data filtering array, and then the target matching data is used as valid target buried data; if the event timestamp of the target matching data is greater than the difference between the current timestamp and the preset time period, it can be considered that the target matching data is not within the filtering time period, and the target matching data is directly deleted from the target array of the running memory.

[0062] Optionally, after the target array is identified, the number of target buried point data in the target array for each event identifier and event result is counted, and the number of buried point data corresponding to each target buried point data is recorded.

[0063] S230, performing data processing on the target matching data according to the number of embedded point data of each target matching data, and determining the data processing result.

[0064] Optionally, in another optional embodiment of the present invention, data processing is performed on the target matching data according to the number of embedded point data of each target matching data to determine the data processing result, including:

[0065] If the number of target buried point data is greater than the preset data volume threshold, the target buried point data is deleted; if the number of target buried point data is less than the preset data volume threshold, the target buried point data is saved and reported according to the data reporting method.

[0066] Among them, the preset data volume threshold can be a threshold information pre-set for identifying duplicate and invalid buried point data. It should be noted that for a target buried point data, within the filtering time period corresponding to the data filtering array, if the number of buried point data of the target buried point data is greater than the preset data volume threshold, the target buried point data can be considered to be duplicate and invalid buried point data; if the number of buried point data of the target buried point data is less than the preset data volume threshold, the target buried point data can be considered to be valid buried point data.

[0067] Optionally, for a target buried point data, if the target buried point data is determined to be duplicate and invalid buried point data, it is necessary to filter the target buried point data and delete the target buried point data in the running memory. If the target buried point data is determined to be valid buried point data, it is necessary to save the target buried point data in the running memory and report the target buried point data through the first reporting method.

[0068] Specifically, data identification is performed on a type of target data in turn. For the target buried data, if the number of buried data items of the target buried data is greater than the preset data volume threshold, the target buried data is deleted from the running memory; if the number of buried data items of the target buried data is less than the preset data volume threshold, the target buried data is saved in the running memory, and the target buried data is reported through the first reporting method.

[0069] The technical solution of the embodiment of the present invention determines the data to be filtered through data filtering, determines the corresponding data filtering method based on the first reporting method, and then filters the data to be filtered in the running memory, which can filter a large amount of repeated and invalid buried data in the running memory, reducing the storage amount of data in the running memory, thereby achieving the problem that the memory will not increase infinitely and cause overflow crash. Solve the technical problem that the buried data collected in the prior art is repeated and invalid. It can effectively enhance the stability of the project and improve the user experience.

[0070] Figure 3 A flowchart of another method for filtering and processing duplicate buried data provided by an embodiment of the present invention. The relationship between this embodiment and the above embodiment is the processing method of the target matching data when the data reporting method corresponding to the target matching data is the second reporting method. Figure 3 As shown, the method includes:

[0071] S310: Filter and match the embedded point data based on the data filter array sent by the server to determine at least one target matching data.

[0072] S320. When the data reporting method of the target matching data is the second reporting method, the number of data items of the target matching data within the filtering time period corresponding to the data filtering array is queried in a preset database to determine the number of embedded point data items corresponding to each target matching data.

[0073] The preset database may be a database pre-set to store the buried point data. It should be noted that the preset database may be any database, such as a local database or a remote database, and the present invention does not limit this.

[0074] Optionally, in an embodiment of the present invention, for the target matching data which is the buried data reported in the second reporting method, that is, the reporting method of the buried data is internal reporting of the project, after the buried data is collected, the buried data will also be stored in a preset database. When each buried data is stored, when the database records the buried data, the event identifier, event result and event timestamp of the buried data will be recorded accordingly.

[0075] Specifically, for the target matching data, when the data reporting method of the target matching data is the first reporting method, the number of data items of each target matching data within the filtering time period corresponding to the data filtering array is queried in the preset database to obtain the number of buried point data items corresponding to each target matching data.

[0076] Optionally, in another optional embodiment of the present invention, querying the number of embedded point data items of target matching data within the filtering time period corresponding to the data filtering array in a preset database includes:

[0077] According to the event identification and event result of the target matching data and the filtering time period, data query is performed in the preset database to determine the target buried point data and the number of buried point data of the target buried point data.

[0078] Specifically, when querying the target matching data in the preset database, determine the filtering time period closest to the current timestamp based on the current timestamp, and use the event identifier and event result of each target matching data and the filtering time period closest to the current timestamp as query conditions in turn. Perform data query in the preset database based on the query conditions. If there is a buried point data with the same event identifier and event result as a target matching data in the database, and the event timestamp of the buried point data is in the filtering data period closest to the current timestamp, then the buried point data is considered to meet the query conditions, and the buried point data is used as the target buried point data, and the number of buried point data items of the target buried point data in the database is recorded. After querying all the target matching data in the database, if there is target buried point data, the database outputs each target buried point data and the number of buried point data items corresponding to each target buried point data; if there is no target buried point data, the database outputs "none".

[0079] S330, process the target matching data according to the number of embedded point data of each target matching data, and determine the data processing result.

[0080] Optionally, in another optional embodiment of the present invention, data processing is performed on the target matching data according to the number of embedded point data of each target matching data to determine the data processing result, including:

[0081] If the number of target buried point data is greater than the preset data volume threshold, the target buried point data is deleted; if the number of target buried point data is less than the preset data volume threshold, the target buried point data is saved and reported according to the data reporting method.

[0082] Optionally, for a target buried point data, within the filtering time period corresponding to the data filtering array, if the number of buried point data items of the target buried point data is greater than the preset data volume threshold, the target buried point data can be considered to be duplicate and invalid buried point data; if the number of buried point data items of the target buried point data is less than the preset data volume threshold, the target buried point data can be considered to be valid buried point data.

[0083] Optionally, for a target buried point data, if the target buried point data is determined to be duplicate and invalid buried point data, it is necessary to filter the target buried point data and delete the target buried point data from the preset database. If the target buried point data is determined to be valid buried point data, it is necessary to save the target buried point data in the preset database and report the target buried point data through the second reporting method.

[0084] Specifically, data identification is performed on a type of target data in turn. For the target buried point data, if the number of buried point data items of the target buried point data is greater than the preset data volume threshold, the target buried point data is deleted from the preset database; if the number of buried point data items of the target buried point data is less than the preset data volume threshold, the target buried point data is saved in the preset database, and the target buried point data is reported through the second reporting method.

[0085] The technical solution of the embodiment of the present invention determines the data to be filtered through data filtering, determines the corresponding data filtering method based on the second reporting method, and then filters the data to be filtered in the running memory, which can filter a large amount of repeated and invalid buried point data in the preset database, reduce the storage amount of data in the preset database, and prevent the problem of reporting a large amount of repeated and invalid buried point data. Solve the technical problem of repeated and invalid buried point data collected in the prior art. It can effectively enhance the stability of the project and improve the user experience.

[0086] Figure 4A schematic diagram of a structure of a device for filtering and processing duplicate buried point data provided by an embodiment of the present invention. Figure 4 As shown, the device includes: a filtering module 410, a query module 420 and a processing module 430; wherein,

[0087] The filtering module 410 is used to filter and match the buried data based on the data filtering array sent by the server to determine at least one target matching data; wherein the buried data includes an event identifier, an event result and an event timestamp;

[0088] The query module 420 is used to obtain the data reporting method of each target matching data, query the number of data items of each target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determine the number of embedded point data items corresponding to each target matching data;

[0089] The processing module 430 is used to perform data processing on the target matching data according to the number of buried data of each target matching data, and determine the data processing result.

[0090] The technical solution of the embodiment of the present invention is to filter and match the buried data based on the data filtering array sent by the server, determine at least one target matching data, obtain the data reporting method of each target matching data, query the number of data items of each target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determine the number of buried data items corresponding to each target matching data; perform data processing on the target matching data according to the number of buried data items of each target matching data, determine the data processing result, determine the data to be filtered through data filtering, and determine the corresponding data filtering method based on different data reporting methods, which can effectively support the operation of the project, and then perform data filtering on the data to be filtered, and can filter a large number of repeated and invalid buried data, so as to better monitor and analyze user behavior; solve the technical problem that the buried data collected in the prior art is repeated and invalid. It can effectively enhance the stability of the project and improve the user experience.

[0091] Optionally, the filtering module 410 is specifically used for:

[0092] The event identifier and the event result of the buried data are filtered and matched based on the filtering rules of the data filtering array sent by the server, and each buried data whose event identifier and the event result match the filtering rules of the data filtering array is determined as the target matching data.

[0093] Optionally, the data reporting method includes a first reporting method and a second reporting method;

[0094] Optionally, the query module 420 is specifically used for: when the data reporting method of the target matching data is the first reporting method, querying the number of data items of the target matching data within the filtering time period corresponding to the data filtering array in the running memory, and determining the number of embedded point data items corresponding to each of the target matching data.

[0095] Optionally, the query module 420 is further specifically used to: store the buried point data in the running memory that has the same event identifier and event result as the target matching data into a preset target array;

[0096] The target matching data in the target array is traversed. If the event timestamp of one of the target matching data in the target array is less than the difference between the current timestamp and the preset time period, the target matching data is determined as the target buried point data, and the number of buried point data corresponding to the target buried point data is recorded.

[0097] Optionally, the query module 420 is also specifically used for: when the data reporting method of the target matching data is the second reporting method, querying the number of data items of the target matching data within the filtering time period corresponding to the data filtering array in a preset database, and determining the number of embedded point data items corresponding to each of the target matching data.

[0098] Optionally, the query module 420 is also specifically used to: perform data query in the preset database according to the event identifier of the target matching data, the event result and the filtering time period, and determine the target buried point data and the number of buried point data of the target buried point data.

[0099] Optionally, the processing module 430 is specifically configured to: if the number of buried point data items of the target buried point data is greater than a preset data volume threshold, delete the target buried point data;

[0100] If the number of target buried point data items is less than a preset data volume threshold, the target buried point data is saved, and the target buried point data is reported according to the data reporting method.

[0101] The filtering and processing device for duplicate buried point data provided in the embodiment of the present invention can execute the filtering and processing method for duplicate buried point data provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0102] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their modes are only examples and are not intended to limit the implementation of the present invention described or required herein.

[0103] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0104] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer grid such as the Internet or various telecommunication grids.

[0105] The processor 11 may be any general or dedicated processing component with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a filtering processing method for duplicate buried point data.

[0106] In some embodiments, the filtering and processing method for duplicate buried point data may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded or installed on the electronic device 10 via the ROM 12 or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the filtering and processing method for duplicate buried point data described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the filtering and processing method for duplicate buried point data in any other appropriate manner (for example, by means of firmware).

[0107] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the modes / operations specified in the flow chart or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0109] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0111] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a grid browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication grid). Examples of communication grids include: a local area network (LAN), a wide area network (WAN), a blockchain grid, and the Internet.

[0112] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication grid. The client and server relationship is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0113] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0114] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the method for filtering duplicate buried data provided by any embodiment of the present invention are implemented. The method includes:

[0115] Filter and match the buried data based on the data filter array sent by the server to determine at least one target matching data; wherein the buried data includes an event identifier, an event result, and an event timestamp;

[0116] Obtaining a data reporting method for each of the target matching data, querying the number of data items of each of the target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determining the number of embedded point data items corresponding to each of the target matching data;

[0117] The target matching data is processed according to the number of embedded data items of each target matching data to determine the data processing result.

[0118] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0119] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0120] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0121] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of grid, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0122] It should be understood by those skilled in the art that the above modules or steps of the present invention can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a grid composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0123] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0124] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for filtering and processing repeated buried point data, characterized in that: include: Filter and match the buried data based on the data filter array sent by the server to determine at least one target matching data; wherein the buried data includes an event identifier, an event result, and an event timestamp; Obtain a data reporting method for each of the target matching data, query the number of data items of each of the target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determine the number of buried point data items corresponding to each of the target matching data; wherein the data reporting method includes a first reporting method and a second reporting method, the first reporting method may be a third-party buried point reporting, and the second reporting method may be a self-developed buried point SDK reporting mechanism within the project; Performing data processing on the target matching data according to the number of embedded point data of each target matching data, and determining a data processing result; The filtering and matching of the embedded point data based on the data filtering array sent by the server to determine at least one target matching data includes: The event identifier and the event result of the buried data are filtered and matched based on the filtering rules of the data filtering array sent by the server, and each buried data whose event identifier and the event result match the filtering rules of the data filtering array is determined as the target matching data.

2. The method according to claim 1, characterized in that The querying the number of data items of each target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determining the number of embedded point data items corresponding to each target matching data, includes: When the data reporting method of the target matching data is the first reporting method, the number of data items of the target matching data within the filtering time period corresponding to the data filtering array is queried in the running memory to determine the number of embedded point data items corresponding to each of the target matching data.

3. The method according to claim 2, characterized in that The step of querying the running memory for the number of embedded point data items of the target matching data within the filtering time period corresponding to the data filtering array includes: The embedded point data having the same event identifier and event result as the target matching data in the running memory is stored in a preset target array; The target matching data in the target array is traversed. If the event timestamp of one of the target matching data in the target array is less than the difference between the current timestamp and the preset time period, the target matching data is determined as the target buried point data, and the number of buried point data corresponding to the target buried point data is recorded.

4. The method according to claim 2, characterized in that: The querying of the number of embedded point data items of the target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method also includes: When the data reporting method of the target matching data is the second reporting method, the number of data items of the target matching data within the filtering time period corresponding to the data filtering array is queried in a preset database to determine the number of embedded point data items corresponding to each of the target matching data.

5. The method according to claim 4, characterized in that The step of querying the preset database for the number of embedded point data items of the target matching data within the filtering time period corresponding to the data filtering array includes: According to the event identifier of the target matching data, the event result and the filtering time period, data query is performed in the preset database to determine the target buried point data and the number of buried point data items of the target buried point data.

6. The method according to claim 3, characterized in that The step of performing data processing on the target matching data according to the number of embedded point data of each target matching data to determine the data processing result includes: If the number of target buried point data is greater than a preset data volume threshold, the target buried point data is deleted; If the number of target buried point data items is less than a preset data volume threshold, the target buried point data is saved, and the target buried point data is reported according to the data reporting method.

7. A filtering and processing device for repeated buried point data, characterized in that: include: A filtering module, used to filter and match the buried data based on the data filtering array sent by the server, and determine at least one target matching data; wherein the buried data includes an event identifier, an event result and an event timestamp; A query module is used to obtain a data reporting method for each of the target matching data, query the number of data items of each of the target matching data within the filtering time period corresponding to the data filtering array based on the data reporting method, and determine the number of buried point data items corresponding to each of the target matching data; wherein the data reporting method includes a first reporting method and a second reporting method, the first reporting method may be a third-party buried point reporting, and the second reporting method may be a self-developed buried point SDK reporting mechanism within the project; A processing module, used to perform data processing on the target matching data according to the number of buried point data of each target matching data, and determine the data processing result; The filtering module is specifically used for: The event identifier and the event result of the buried data are filtered and matched based on the filtering rules of the data filtering array sent by the server, and each buried data whose event identifier and the event result match the filtering rules of the data filtering array is determined as the target matching data.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the filtering and processing method for duplicate buried point data described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the filtering and processing method for duplicate buried point data described in any one of claims 1-6 when executed.

Images