Evaluation method, device and equipment, computer readable storage medium and program product

Abstract

The embodiment of the invention provides an evaluation method, device and equipment, a computer readable storage medium and a program product, and relates to the fields of burying points, maps and the like, and application scenes include but are not limited to scenes for evaluating the quality of burying point events processed by a software development kit. The method comprises the steps of obtaining a point burying event set for an application program and a point burying event quantity corresponding to the point burying event set in a preset time period through a software development kit called by the application program in a target terminal, wherein the point burying event quantity is used for representing the total quantity of all point burying events in the point burying event set; based on the burying point event set, determining a real event quantity and a filtering event quantity through a software development kit; and sending the burying point event quantity, the real event quantity and the filtering event quantity to a server through the software development kit, so that the server evaluates the burying point event processing quality of the software development kit based on the burying point event quantity, the real event quantity and the filtering event quantity corresponding to each terminal in the terminal set.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly, the present disclosure relates to an evaluation method, device, equipment, computer readable storage medium and program product. BACKGROUND

[0002] In the prior art, an APP (Application) on a terminal calls an SDK (Software Development Kit) to collect a set of events during running of the APP, the SDK processes the set of events (for example, the SDK determines whether the set of events meets a rule), and sends the set of events meeting the rule to a server, and the server evaluates the quality of the SDK processing the set of events based on a change in the number of the set of events meeting the rule; in this way, the accuracy of the evaluation of the quality of the SDK processing the set of events is low. For example, after a new version of the APP is released, the server cannot determine whether a sharp drop in the number of the set of events meeting the rule is caused by a change in business or a technical fault, such as loss of the set of events in the process of the SDK processing the set of events or loss of parameters of the set of events in the process of the SDK processing the set of events; in this way, the accuracy of the evaluation of the quality of the SDK processing the set of events is low. SUMMARY

[0003] The present disclosure proposes an evaluation method, device, equipment, computer readable storage medium and computer program product to solve the problem of how to improve the accuracy of the evaluation of the quality of an SDK processing a set of events.

[0004] In a first aspect, the present disclosure provides an evaluation method, executed by a target terminal, comprising: obtaining, by an SDK called by an application program in the target terminal, a set of events for the application program and a quantity of the set of events in a preset time period, the quantity of the set of events representing a total number of all events in the set of events; determining, by the SDK, a real quantity of events and a filtered quantity of events based on the set of events, the real quantity of events representing a number of events in the set of events meeting a preset first rule, and the filtered quantity of events representing a number of events in the set of events not meeting the preset first rule; sending, by the SDK, the quantity of the set of events, the real quantity of events and the filtered quantity of events to a server, so that the server evaluates the quality of the SDK processing the set of events based on the quantity of the set of events, the real quantity of events and the filtered quantity of events corresponding to each terminal in a set of terminals, the set of terminals including the target terminal, and each terminal in the set of terminals including the SDK.

[0005] In one embodiment, before obtaining the set of events and the amount of events corresponding to the set of events of the application program within the preset time period through the software development kit called by the application program in the target terminal, the method further comprises: initializing the original software development kit in the target terminal to obtain the software development kit, and generating a terminal identifier, an event count identifier, a real event count identifier and a filtered event count identifier of the target terminal; obtaining the amount of events corresponding to the set of events through the software development kit called by the application program in the target terminal, comprising: determining, through the software development kit called by the application program in the target terminal, a first count value corresponding to the event count identifier and a second count value corresponding to the event count identifier, the first count value representing the event count at the first time point, and the second count value representing the event count at the second time point, the first time point being the starting time point of the preset time period, and the second time point being the ending time point of the preset time period; determining the amount of events corresponding to the set of events based on the first count value and the second count value.

[0006] In one embodiment, based on the set of events, the software development kit determines the amount of real events and the amount of filtered events, comprising: based on the set of events and a preset first rule, the software development kit determines a set of real events and a set of filtered events; the software development kit counts the set of real events to determine the amount of real events, and counts the set of filtered events to determine the amount of filtered events, each real event in the set of real events representing an event that meets the preset first rule, and each filtered event in the set of filtered events representing an event that does not meet the preset first rule.

[0007] In one embodiment, the software development kit counts the set of real events to determine the amount of real events, comprising: the software development kit counts the set of real events to determine a third count value corresponding to the real event count identifier and a fourth count value corresponding to the real event count identifier, the third count value representing the real event count at the first time point, and the fourth count value representing the real event count at the second time point, the first time point being the starting time point of the preset time period, and the second time point being the ending time point of the preset time period; determining the amount of real events corresponding to the set of real events based on the third count value and the fourth count value.

[0008] In one embodiment, the counting processing for the filtered event set is performed by the software development kit, the filtered event amount is determined, comprising: The counting processing for the filtered event set is performed by the software development kit, the fifth count value corresponding to the filtered event count identifier and the sixth count value corresponding to the filtered event count identifier are determined, the fifth count value is used to represent the filtered event count at the first time point, the sixth count value is used to represent the filtered event count at the second time point, the first time point is the starting time point of the preset time period, and the second time point is the ending time point of the preset time period; Based on the fifth count value and the sixth count value, the filtered event amount corresponding to the filtered event set is determined.

[0009] In one embodiment, before the buried point event amount, the real event amount and the filtered event amount are sent to the server by the software development kit, further comprising: Based on the buried point event amount, the real event amount, the filtered event amount, the parameters in the buried point events corresponding to the buried point event amount and the filtering reasons of the filtered events corresponding to the filtered event amount, the service level agreement buried point event is determined. Based on the parameter values of the parameters in the real events corresponding to the real event amount and the preset second rule, the parameter values meeting the preset second rule and the target real event amount are determined by the software development kit, the target real event amount is used to represent the number of real events corresponding to the parameter values meeting the preset second rule.

[0010] In one embodiment, the buried point event amount, the real event amount and the filtered event amount are sent to the server by the software development kit, comprising: The service level agreement buried point event is sent to the server by the software development kit, so that the server determines the buried point event collection rate based on the service level agreement buried point event. The target real event amount, the real event amount and the parameters in the buried point events corresponding to the buried point event amount are sent to the server by the software development kit, so that the server determines the parameter collection rate based on the target real event amount, the real event amount and the parameters in the buried point events corresponding to the buried point event amount.

[0011] In a second aspect, the present disclosure provides an evaluation method, which is executed by a server, comprising: The buried point event amount, the real event amount and the filtered event amount sent by the software development kit called by the application program in each terminal of the terminal set are received; Based on the buried point event amount, the real event amount and the filtered event amount sent by each terminal, the buried point event collection rate and the parameter collection rate for the software development kit are determined. Based on the buried point event collection rate and the parameter collection rate for the software development kit, the quality score of the software development kit processing the buried point event is determined.

[0012] In one embodiment, based on the amount of the buried point events, the amount of the real events and the amount of the filtered events sent by each terminal, the buried point event collection rate and the parameter collection rate for the software development kit are determined, comprising: Grouping the terminals based on the parameters in the buried point events sent by each terminal, to determine at least one first group; Determining the buried point event collection rate corresponding to the terminals included in each first group in the at least one first group; Based on the buried point event collection rate corresponding to the terminals included in each first group in the at least one first group, determining the buried point event collection rate corresponding to each first group, and determining the buried point event collection rate corresponding to each first group as the buried point event collection rate for the software development kit; Based on the terminal identifiers in the buried point events sent by each terminal, performing sampling processing to determine a plurality of terminals from the terminals, and grouping the plurality of terminals based on the parameters in the buried point events sent by the plurality of terminals, to determine at least one second group; Determining the parameter collection rate corresponding to the terminals included in each second group in the at least one second group; Based on the parameter collection rate corresponding to the terminals included in each second group in the at least one second group, determining the parameter collection rate corresponding to each second group, and determining the parameter collection rate corresponding to each second group as the parameter collection rate for the software development kit.

[0013] In one embodiment, based on the device identifiers in the buried point events sent by each terminal, performing sampling processing to determine a plurality of terminals from the terminals, comprising: Based on the terminal identifiers in the buried point events sent by each terminal and a preset sampling rate, determining a plurality of terminals from the terminals by using a string hash algorithm.

[0014] In a third aspect, the present disclosure provides an evaluation device applied to a target terminal, comprising: A first processing module configured to acquire, by a software development kit called by an application program in the target terminal, a set of buried point events for the application program and an amount of the buried point events corresponding to the set of buried point events within a preset time period, the amount of the buried point events being used to represent the total number of all the buried point events in the set of buried point events; A second processing module configured to determine, by the software development kit based on the set of buried point events, an amount of real events and an amount of filtered events, the amount of the real events being used to represent the number of the buried point events in the set of buried point events that meet a preset first rule, and the amount of the filtered events being used to represent the number of the buried point events in the set of buried point events that do not meet the preset first rule; The third processing module is configured to send the buried point event quantity, the real event quantity and the filtered event quantity to the server through the software development kit, so that the server evaluates the quality of the software development kit in processing the buried point event based on the buried point event quantity, the real event quantity and the filtered event quantity corresponding to each terminal in the terminal set, and the terminal set includes the target terminal and the software development kit in each terminal of the terminal set.

[0015] In a fourth aspect, the present disclosure provides an evaluation device applied to a server, comprising: The fourth processing module is configured to receive the buried point event quantity, the real event quantity and the filtered event quantity sent by the software development kit called by the application program in each terminal of the terminal set; The fifth processing module is configured to determine the buried point event collection rate and the parameter collection rate of the software development kit based on the buried point event quantity, the real event quantity and the filtered event quantity sent by each terminal; The sixth processing module is configured to determine the quality score of the software development kit in processing the buried point event based on the buried point event collection rate and the parameter collection rate of the software development kit.

[0016] In a fifth aspect, the present disclosure provides an electronic device, comprising a processor, a memory and a bus; The bus is configured to connect the processor and the memory; The memory is configured to store operation instructions; The processor is configured to execute the evaluation method of the first aspect or the second aspect of the present disclosure by calling the operation instructions.

[0017] In a sixth aspect, the present disclosure provides a computer readable storage medium storing a computer program, and the computer program is used to execute the evaluation method of the first aspect or the second aspect of the present disclosure.

[0018] In a seventh aspect, the present disclosure provides a computer program product comprising a computer program, and the computer program is executed by the processor to implement the steps of the evaluation method in the first aspect or the second aspect of the present disclosure.

[0019] The technical solution provided by the embodiments of the present disclosure has at least the following beneficial effects: The software development kit called by the application program in the target terminal is used to obtain a set of buried point events for the application program in a preset time period and a buried point event quantity corresponding to the set of buried point events, and the buried point event quantity is used to represent the total number of all buried point events in the set of buried point events; based on the set of buried point events, the software development kit is used to determine a real event quantity and a filtered event quantity, the real event quantity is used to represent the number of buried point events in the set of buried point events that meet a preset first rule, and the filtered event quantity is used to represent the number of buried point events in the set of buried point events that do not meet the preset first rule; the software development kit is used to send the buried point event quantity, the real event quantity and the filtered event quantity to the server, so that the server evaluates the quality of the software development kit in processing the buried point events based on the buried point event quantity, the real event quantity and the filtered event quantity corresponding to each terminal in the set of terminals; in this way, the terminal determines the buried point event quantity, the real event quantity and the filtered event quantity through the software development kit, so that the server evaluates the quality of the software development kit in processing the buried point events based on the buried point event quantity, the real event quantity and the filtered event quantity; for example, the server determines a buried point event collection rate and a parameter collection rate based on the buried point event quantity, the real event quantity and the filtered event quantity, the server evaluates whether the buried point events are lost in the process of the software development kit processing the buried point events through the event collection rate, and evaluates whether the parameters of the buried point events are lost in the process of the software development kit processing the buried point events through the parameter collection rate; for another example, after the software development kit releases a new version, business changes will not cause a large range of changes in the buried point event collection rate and the parameter collection rate, while technical faults will cause a large range of changes in the buried point event collection rate and the parameter collection rate, therefore, the influence of business changes can be excluded through the buried point event collection rate and the parameter collection rate, and it can be accurately judged whether a technical fault occurs in the process of the software development kit processing the buried point events; thereby the accuracy of evaluating the quality of the software development kit in processing the buried point events is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings needed to be used in the description of the embodiments of the present disclosure will be briefly introduced.

[0021] Figure 1 An architecture schematic diagram of an evaluation system provided by the embodiments of the present disclosure is provided. Figure 2 A flow schematic diagram of an evaluation method provided by the embodiments of the present disclosure is provided. Figure 3 An evaluation schematic diagram provided by the embodiments of the present disclosure is provided. Figure 4 A flow schematic diagram of an evaluation method provided by the embodiments of the present disclosure is provided. Figure 5 A flow schematic diagram of an evaluation method provided by the embodiments of the present disclosure is provided. Figure 6 A structural schematic diagram of an evaluation device provided by an embodiment of the present disclosure is shown in FIG. 1. Figure 7 A structural schematic diagram of an evaluation device provided by an embodiment of the present disclosure is shown in FIG. 1. Figure 8 A structural schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown in FIG. 1. DETAILED DESCRIPTION

[0022] Embodiments of the present disclosure will be described below in conjunction with the accompanying drawings. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions of the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions of the embodiments of the present disclosure.

[0023] Those skilled in the art can understand that the singular forms "a", "an" and "the" used herein include plural forms unless specifically stated otherwise. It should be further understood that the terms "comprise" and "include" used by the embodiments of the present disclosure mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the present technology. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or can mean that the element and the other element establish a connection relationship through an intermediate element. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The term "and / or" used herein indicates that at least one of the items defined by the term, for example, "A and / or B" indicates implementation as "A", or implementation as "B", or implementation as "A and B".

[0024] It can be understood that in the specific embodiments of the present disclosure, data related to evaluation is involved, and when the above embodiments of the present disclosure are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0025] In order to make the purposes, technical solutions and advantages of the present disclosure clearer, the embodiments of the present disclosure will be described in further detail below in conjunction with the accompanying drawings.

[0026] The embodiments of the present disclosure are an evaluation method provided by an evaluation system, which relates to the fields of burying points and maps.

[0027] In order to better understand and illustrate the scheme of the embodiments of the present disclosure, some technical terms involved in the embodiments of the present disclosure will be briefly described below.

[0028] Event tracking: Event tracking is an action, that is, the action of creating an event tracking point at a certain time. An event tracking point can be a record that records the time. For example, "user login" is an event, and a record is generated accordingly: "when, where, who performed what action". This record is an event tracking point. Events include APP launch, page exposure, element click, etc.

[0029] Event tracking: Code snippets inserted into an application or website to collect relevant data; by inserting tracking code at specific interaction points or key operations, developers can record user behavior, clicks, browsing, and other events for data analysis and user behavior analysis.

[0030] String hashing algorithm: A string hashing algorithm is an algorithm that converts a string of arbitrary length into a hash value of fixed length.

[0031] The existing technologies for evaluating the quality of software development kits in handling event tracking have the following problems: Existing technologies typically use online data directly (such as rule-compliant event tracking), focusing on the year-on-year or month-on-month comparison of the absolute quantity of online data. If there are significant differences in the year-on-year or month-on-month comparisons, it can reflect problems with the online data to some extent. However, the biggest drawback of existing technologies is that they are affected by business logic. For example, after an app releases a new version, the number of rule-compliant event tracking points may plummet. It is impossible to determine whether the plummeting number is caused by business changes or technical failures. Technical failures may include the loss of event tracking points or the loss of event parameters during the processing of event tracking points by the software development kit (SDK). As a result, it is impossible to accurately evaluate the quality of event tracking point processing by the SSD, leading to low accuracy in evaluating the quality of event tracking point processing by the SSD.

[0032] Based on this, the present disclosure provides an evaluation method, apparatus, device, computer-readable storage medium, and program product, and the specific technical solutions will be described in detail below.

[0033] The solutions provided in this disclosure relate to evaluation technology. The technical solutions of this disclosure will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0034] To better understand the solution provided in this disclosure, the solution will be described below in conjunction with a specific application scenario.

[0035] In one embodiment, Figure 1The diagram shows an architecture schematic of an evaluation system applicable to embodiments of this disclosure. It is understood that the evaluation methods provided in these embodiments can be applied to, but are not limited to, applications such as... Figure 1 In the application scenarios shown.

[0036] In this example, as Figure 1 As shown, the architecture of the evaluation system in this example may include, but is not limited to, server 10, terminal 20, and database 30. Server 10, terminal 20, and database 30 can interact with each other via network 40.

[0037] The application in terminal 20 calls a software development kit (SDK) to obtain a set of event tracking data for the application within a preset time period and the corresponding number of event tracking data points. The number of event tracking data points represents the total number of all event tracking data points in the set. Based on the event tracking data set, the SSD determines the number of actual event tracking data points and the number of filtered event tracking data points. The number of actual event tracking data points represents the number of event tracking data points in the set that conform to a preset first rule, and the number of filtered event tracking data points represents the number of event tracking data points in the set that do not conform to the preset first rule. The SSD sends the number of event tracking data points, the number of actual event tracking data points, and the number of filtered event tracking data points to the server, so that the server 10 can evaluate the quality of the SSD's handling of event tracking data points based on the number of event tracking data points, the number of actual event tracking data points, and the number of filtered event tracking data points corresponding to each terminal in the terminal set. The server 10 stores the number of event tracking data points, the number of actual event tracking data points, and the number of filtered event tracking data points corresponding to each terminal in the terminal set in the database 30. Each terminal 20 in the terminal set includes a SSD.

[0038] It is understood that the above is only one example, and this embodiment is not limited here.

[0039] Terminals include, but are not limited to, smartphones (such as Android phones, iOS phones, etc.), mobile phone emulators, tablets, laptops, digital broadcast receivers, MIDs (Mobile Internet Devices), PDAs (Personal Digital Assistants), smart voice interaction devices, smart home appliances, and in-vehicle terminals.

[0040] A server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0041] The network can include, but is not limited to, a wired network including a local area network, a metropolitan area network, and a wide area network, and a wireless network including Bluetooth, Wi-Fi, and other networks that enable wireless communication. The actual application scenario requirements can also be determined, and are not limited here.

[0042] Referring to Figure 2 , Figure 2 A flowchart of an evaluation method provided by an embodiment of the present disclosure is shown, which can be executed by any electronic device, such as a terminal. As an optional implementation, the method can be executed by a target terminal. For the convenience of description, in the description of some optional embodiments below, the target terminal will be taken as an example to illustrate the execution subject of the method. As shown in Figure 2 The evaluation method provided by the embodiment of the present disclosure includes the following steps: S201, obtaining, by a software development kit called by an application program in a target terminal, a set of buried point events for the application program in a preset time period and a buried point event quantity corresponding to the set of buried point events, the buried point event quantity being used to represent the total number of all buried point events in the set of buried point events.

[0043] Specifically, an APP (application program) is installed on the terminal, such as a game APP, a video APP, an entertainment APP, etc.; SDK (software development kit) is a software program package, which encapsulates specific functional logic; for example, the SDK is installed in the APP.

[0044] The preset time period is, for example, 1 minute. For example, the SDK will count the buried point event quantity, the real event quantity, and the filtered event quantity per minute.

[0045] For example, the original software development kit in the target terminal is initialized to obtain a software development kit, and the software development kit called by the application program in the target terminal is used to obtain a set of buried point events for the application program in a preset time period based on a buried point event collection interface, the set of buried point events including a plurality of buried point events.

[0046] S202, determining, by the software development kit, a real event quantity and a filtered event quantity based on the set of buried point events, the real event quantity being used to represent the number of buried point events in the set of buried point events that meet a preset first rule, and the filtered event quantity being used to represent the number of buried point events in the set of buried point events that do not meet the preset first rule.

[0047] Specifically, the real event quantity includes a plurality of real events, each of the plurality of real events being a trace event meeting a preset first rule; the filtered event quantity includes a plurality of filtered events, each of the plurality of filtered events being a trace event not meeting the preset first rule; the trace event includes a parameter of the trace event, and the parameter of the trace event includes a common parameter and a non-common parameter, the common parameter including a business identifier, a trace event reporting date, an APP version number, a terminal identifier, etc., and the non-common parameter including an exposure duration, an exposure area, an element name, a touch manner, etc.

[0048] The preset first rule includes a parameter value of the non-common parameter, i.e., the parameter value of the non-common parameter is used to determine whether the trace event meets the preset first rule. For example, the parameter value of the non-common parameter is that the exposure area is greater than 1 / 3 of the component area or the exposure duration is greater than 1 second; the preset first rule is, for example, that the exposure area is greater than 1 / 3 of the component area and the exposure duration is greater than 1 second. If the non-common parameter in a certain trace event meets the preset first rule, the trace event is determined to be a real event; if the non-common parameter in a certain trace event does not meet the preset first rule, the trace event is determined to be a filtered event, and the filtered event is discarded.

[0049] In S203, the SDK sends the trace event quantity, the real event quantity and the filtered event quantity to the server, so that the server evaluates the quality of the SDK processing the trace event based on the trace event quantity, the real event quantity and the filtered event quantity corresponding to each terminal in a terminal set, the terminal set including the target terminal, and each terminal in the terminal set including the SDK.

[0050] Specifically, the SDK sends the trace event quantity, the real event quantity and the filtered event quantity to the server, and the server determines a trace event collection rate and a parameter collection rate based on the trace event quantity, the real event quantity and the filtered event quantity. The server evaluates whether the trace event is lost in the process of the SDK processing the trace event through the trace event collection rate, and evaluates whether the parameter of the trace event is lost in the process of the SDK processing the trace event through the parameter collection rate.

[0051] In the embodiments of the present disclosure, the software development kit called by the application program in the target terminal is used to obtain a set of buried point events of the application program in a preset time period and a buried point event quantity corresponding to the set of buried point events, and the buried point event quantity is used to represent the total number of all buried point events in the set of buried point events; based on the set of buried point events, the software development kit is used to determine a real event quantity and a filtered event quantity, the real event quantity is used to represent the number of buried point events in the set of buried point events that meet a preset first rule, and the filtered event quantity is used to represent the number of buried point events in the set of buried point events that do not meet the preset first rule; the software development kit is used to send the buried point event quantity, the real event quantity and the filtered event quantity to a server, so that the server evaluates the quality of the software development kit in processing the buried point events based on the buried point event quantity, the real event quantity and the filtered event quantity corresponding to each terminal in a terminal set; in this way, the terminal determines the buried point event quantity, the real event quantity and the filtered event quantity through the software development kit, so that the server evaluates the quality of the software development kit in processing the buried point events based on the buried point event quantity, the real event quantity and the filtered event quantity; for example, the server determines a buried point event collection rate and a parameter collection rate based on the buried point event quantity, the real event quantity and the filtered event quantity, the server evaluates whether the buried point events are lost in the process of the software development kit processing the buried point events through the event collection rate, and evaluates whether the parameters of the buried point events are lost in the process of the software development kit processing the buried point events through the parameter collection rate; for another example, after the software development kit releases a new version, business changes will not cause a large range change of the buried point event collection rate and the parameter collection rate, while a technical fault will cause a large range change of the buried point event collection rate and the parameter collection rate, therefore, the influence of business changes can be excluded through the buried point event collection rate and the parameter collection rate, and whether a technical fault occurs in the process of the software development kit processing the buried point events can be accurately judged; thereby the accuracy of evaluating the quality of the software development kit in processing the buried point events is improved.

[0052] In one embodiment, before the software development kit called by the application program in the target terminal is used to obtain a set of buried point events of the application program in a preset time period and a buried point event quantity corresponding to the set of buried point events, the method further includes: initializing the original software development kit in the target terminal to obtain the software development kit, and generating a terminal identifier of the target terminal, a buried point event count identifier, a real event count identifier and a filtered event count identifier; The software development kit called by the application program in the target terminal is used to obtain a buried point event quantity corresponding to the set of buried point events, including: The software development kit called by the application program in the target terminal determines a first count value corresponding to the buried point event count identifier and a second count value corresponding to the buried point event count identifier, the first count value being used to represent the buried point event count at a first time point, and the second count value being used to represent the buried point event count at a second time point, the first time point being a starting time point of a preset time period, and the second time point being an ending time point of the preset time period; Based on the first count value and the second count value, the buried point event quantity corresponding to the buried point event set is determined.

[0053] Specifically, for example, the original SDK (original software development kit) in the terminal is initialized to obtain the SDK (software development kit), and the terminal ID (Identity Document, identity) of the terminal, the buried point event count ID (buried point event count identifier), the real event count ID (real event count identifier), the filtered event count ID (filtered event count identifier), and the like of the terminal are generated; wherein, as shown in Figure 3 The SDK starts collecting buried point events, and calls an interface (buried point event collection interface) for buried point event counting, the buried point event count ID being the call interface count ID, and the buried point event quantity being the call interface quantity; as shown in Figure 3 The SDK performs real event reporting counting, and the real event count ID is the real event reporting count ID.

[0054] For example, the SDK starts a timing polling task, which runs every 1 minute, that is, the preset time period is 1 minute, the first time point is the starting time point of 1 minute (for example, 12:01), and the second time point is the ending time point of 1 minute (for example, 12:02), the SDK calls the interface (buried point event collection interface) for buried point event counting, the initial value corresponding to the buried point event count ID is the first count value, which is counted once every 1 minute, the buried point event count ID+1, that is, the first count value is accumulated by 1, until the end of 1 minute, the terminal value corresponding to the buried point event count ID is obtained, that is, the second count value; the first count value corresponding to the buried point event count ID is used to represent the buried point event count at the first time point (for example, 12:01), and the second count value corresponding to the buried point event count ID is used to represent the buried point event count at the second time point (for example, 12:02), the difference between the second count value and the first count value is obtained to obtain the buried point event quantity in 1 minute.

[0055] It should be noted that, for example, the terminal ID is persistently stored in the cache, and after the terminal is started each time, it is checked whether there is a terminal ID in the cache, if there is no terminal ID, a new terminal ID is created.

[0056] In one embodiment, based on the buried point event set, the real event quantity and the filtered event quantity are determined through the software development kit, comprising: Based on the set of buried point events and the preset first rule, a set of real events and a set of filtered events are determined through a software development kit (SDK); The SDK performs counting on the set of real events to determine the amount of real events, and performs counting on the set of filtered events to determine the amount of filtered events. Each real event in the set of real events represents a buried point event that meets the preset first rule. Each filtered event in the set of filtered events represents a buried point event that does not meet the preset first rule.

[0057] Specifically, for example, as shown in Figure 3 the buried point event processing flow (process of the SDK processing the buried point event) includes the SDK determining whether the buried point event is compliant, i.e., the SDK determining whether the buried point event meets the preset first rule. If the SDK determines that the buried point event meets the preset first rule, the buried point event is determined to be a real event, and the real event is reported to a first backend server. At the same time, real event reporting counting, i.e., real event counting, is performed. If the SDK determines that the buried point event does not meet the preset first rule, the buried point event is determined to be a filtered event, and the filtered event is discarded. At the same time, filtered event counting is performed.

[0058] For example, the SDK starts a timing polling task that runs every 1 minute, i.e., the preset time period is 1 minute, the first time point is the start time point of 1 minute (e.g., 12:01), and the second time point is the end time point of 1 minute (e.g., 12:02). The SDK performs real event reporting counting, i.e., real event counting. The initial value corresponding to the real event counting ID is a third counting value. The real event counting ID is incremented by 1 every time the counting is performed within 1 minute, i.e., the third counting value is incremented by 1, until the end of 1 minute. The end value corresponding to the real event counting ID, i.e., a fourth counting value, is obtained. The third counting value corresponding to the real event counting ID represents the real event counting at the first time point (e.g., 12:01). The fourth counting value corresponding to the real event counting ID represents the real event counting at the second time point (e.g., 12:02). The difference between the fourth counting value and the third counting value is obtained to determine the amount of real events within 1 minute.

[0059] For example, the SDK starts a timing polling task, which runs every 1 minute, i.e., the preset time period is 1 minute, the first time point is the start time point of 1 minute (e.g., 12:01), the second time point is the end time point of 1 minute (e.g., 12:02), the SDK performs filtered event counting, the initial value corresponding to the filtered event counting ID is the fifth counting value, the filtered event counting ID + 1, i.e., the fifth counting value is accumulated by 1 every time in 1 minute, until the end of 1 minute, to obtain the end value corresponding to the filtered event counting ID, i.e., the sixth counting value; the fifth counting value corresponding to the filtered event counting ID is used to represent the filtered event count at the first time point (e.g., 12:01), and the sixth counting value corresponding to the filtered event counting ID is used to represent the filtered event count at the second time point (e.g., 12:02). The difference between the fifth counting value and the sixth counting value is obtained to obtain the filtered event amount in 1 minute.

[0060] In one embodiment, the SDK performs counting processing on the real event set to determine the real event amount, including: The SDK performs counting processing on the real event set to determine the third counting value corresponding to the real event counting ID and the fourth counting value corresponding to the real event counting ID, the third counting value is used to represent the real event count at the first time point, and the fourth counting value is used to represent the real event count at the second time point; the first time point is the start time point of the preset time period, and the second time point is the end time point of the preset time period. Based on the third counting value and the fourth counting value, the real event amount corresponding to the real event set is determined.

[0061] For example, the SDK starts a timing polling task, which runs every 1 minute, i.e., the preset time period is 1 minute, the first time point is the start time point of 1 minute (e.g., 12:01), the second time point is the end time point of 1 minute (e.g., 12:02), the SDK performs real event reporting counting, i.e., real event counting, the initial value corresponding to the real event counting ID is the third counting value, the real event counting ID + 1, i.e., the third counting value is accumulated by 1 every time in 1 minute, until the end of 1 minute, to obtain the end value corresponding to the real event counting ID, i.e., the fourth counting value; the third counting value corresponding to the real event counting ID is used to represent the real event count at the first time point (e.g., 12:01), and the fourth counting value corresponding to the real event counting ID is used to represent the real event count at the second time point (e.g., 12:02). The difference between the fourth counting value and the third counting value is obtained to obtain the real event amount in 1 minute.

[0062] In one embodiment, the SDK performs counting processing on the filtered event set to determine the filtered event amount, including: The counting processing is performed on the filtered event set through the software development kit, and a fifth count value corresponding to a filtered event count identifier and a sixth count value corresponding to the filtered event count identifier are determined, the fifth count value being used to represent the filtered event count at a first time point, and the sixth count value being used to represent the filtered event count at a second time point, the first time point being a starting time point of a preset time period, and the second time point being an ending time point of the preset time period; Based on the fifth count value and the sixth count value, the filtered event quantity corresponding to the filtered event set is determined.

[0063] Specifically, for example, the SDK starts a timing polling task, which is run every 1 minute, that is, the preset time period is 1 minute, the first time point is the starting time point of 1 minute (for example, 12:01), and the second time point is the ending time point of 1 minute (for example, 12:02), the SDK performs filtered event counting, the initial value corresponding to the filtered event count ID is the fifth count value, which is counted once every 1 minute, the filtered event count ID+1, that is, the fifth count value is accumulated by 1, until the end of 1 minute, the terminal value corresponding to the filtered event count ID is obtained, that is, the sixth count value; the fifth count value corresponding to the filtered event count ID is used to represent the filtered event count at the first time point (for example, 12:01), and the sixth count value corresponding to the filtered event count ID is used to represent the filtered event count at the second time point (for example, 12:02), the difference between the fifth count value and the sixth count value is obtained, and the filtered event quantity in 1 minute is obtained.

[0064] In one embodiment, before the SDK sends the quantity of the buried point event, the quantity of the real event and the quantity of the filtered event to the server, it further includes: Based on the quantity of the buried point event, the quantity of the real event, the quantity of the filtered event, the parameters in the buried point event corresponding to the quantity of the buried point event and the filtering reason of the filtered event corresponding to the quantity of the filtered event, the service level agreement buried point event is determined. Based on the parameter values of the parameters in the real events corresponding to the quantity of the real event and a preset second rule, the SDK determines the parameter values meeting the preset second rule and a target quantity of real events, the target quantity of real events being used to represent the number of real events corresponding to the parameter values meeting the preset second rule.

[0065] Specifically, for example, as Figure 3As shown, the SDK triggers an SLA (Service Level Agreement) event reporting process every minute via a timer. The SLA event reporting process includes: the SDK determining the number of event tracks, the number of actual events, and the number of filtered events, which are the parameters of the SLA event tracks; constructing an SLA event from the number of event tracks, the number of actual events, the number of filtered events, the common parameters in the event tracks corresponding to the number of event tracks, and the filtering reasons for the event tracks corresponding to the number of event tracks; sending the SLA event to the server, which is the second backend server; and the server determining the event collection rate based on the service level agreement event tracks.

[0066] For example, based on the parameter values ​​of the parameters in each real event corresponding to the real event volume and a preset second rule, the SDK determines the parameter values ​​and target real event volume that conform to the preset second rule; the preset second rule, for example, states that if a parameter value is neither empty nor zero, then that parameter value represents a valid parameter value; the SDK sends the target real event volume, the real event volume, and the common parameters in the tracking events corresponding to the tracking event volume to the server, which is as follows: Figure 3 The second backend server shown in the figure determines the parameter collection rate based on the common parameters in the event tracking corresponding to the target real event volume, the real event volume, and the event tracking volume.

[0067] In one embodiment, sending the number of tracked events, the number of real events, and the number of filtered events to the server via a software development kit includes: The service level agreement (SLA) event tracking is sent to the server using a software development kit (SDK), so that the server can determine the event collection rate based on the SLA event tracking. The software development kit (SDK) is used to send the target real event volume, the real event volume, and the parameters in the corresponding tracked events to the server, so that the server can determine the parameter collection rate based on the target real event volume, the real event volume, and the parameters in the corresponding tracked events.

[0068] Specifically, for example, the server determines the event collection rate based on the service level agreement (SLA) for event tracking, and determines the parameter collection rate based on the target real event volume, the real event volume, and the common parameters in the event tracking corresponding to the event tracking volume. The server uses the event collection rate to evaluate whether the event tracking is lost during the process of the software development kit (SDK) processing the event tracking, and uses the parameter collection rate to evaluate whether the parameters of the event tracking are lost during the process of the software development kit (SDK) processing the event tracking.

[0069] For example, the SDK sends 1000 SLA event tracking points to the server from 1000 terminals. The server groups these 1000 SLA event tracking points based on common parameters (such as service identifier, event reporting date, APP version number, etc.), and assigns terminals with the same common parameters to one group. For example, the 1000 terminals are divided into two groups, namely Group A and Group B. Group A includes 3 terminals, and Group B includes 997 terminals. The server determines the corresponding SLA event tracking points for the 3 terminals in Group A based on formula (1). Point event collection rate A, tracking event collection rate B, and tracking event collection rate C; the server calculates the average value among tracking event collection rates A, B, and C, and determines this average value as the tracking event collection rate corresponding to the first group A; based on the same calculation method, the server determines the tracking event collection rate corresponding to the first group B; the server evaluates whether tracking events are lost during SDK processing of tracking events by using the tracking event collection rates corresponding to the first group A and the tracking event collection rates corresponding to the first group B; formula (1) is shown below: Event collection rate = (Actual event count + Filtered event count) / Event count (Formula 1) In formula (1), the number of embedded events, the number of real events, and the number of filtered events are respectively the number of embedded events, the number of real events, and the number of filtered events in SLA embedded events, and the embedded event collection rate is, for example, the embedded event collection rate A.

[0070] For example, the SDKs of 1000 terminals send common parameters from the event tracking data corresponding to the target real event volume, the real event volume, and the event tracking data volume to the server. The server samples from these terminals based on the common parameters sent by the SDKs (e.g., terminal identifier) ​​at a preset sampling rate of 1%, selecting 10 terminals as samples. The server then groups these 10 terminals based on the common parameters sent by the SDKs (e.g., business identifier, event tracking report date, APP version number, etc.), grouping terminals with the same common parameters together. For example, these 10 terminals can be divided into two groups: Group A and Group B. Group A... The first group includes 3 terminals, and the second group B includes 7 terminals. Based on formula (2), the server determines the parameter acquisition rate A, parameter acquisition rate B, and parameter acquisition rate C corresponding to the 3 terminals in the second group A respectively. The server calculates the average value between parameter acquisition rate A, parameter acquisition rate B, and parameter acquisition rate C, and determines this average value as the parameter acquisition rate corresponding to the second group A. Based on the same calculation method, the server determines the parameter acquisition rate corresponding to the second group B. The server evaluates whether the parameters of the event are lost during the SDK processing of the event by using the parameter acquisition rate corresponding to the second group A and the parameter acquisition rate corresponding to the second group B. Formula (2) is shown below: Parameter collection rate = target real event amount / real event amount Formula (2) The target real event amount is used to represent the number of real events corresponding to the parameter value meeting the preset second rule, and the parameter collection rate is, for example, parameter collection rate A.

[0071] The application of the embodiments of the present disclosure has at least the following beneficial effects: The terminal determines the amount of buried events, the amount of real events, and the amount of filtered events through the software development kit, so that the server evaluates the quality of the software development kit processing the buried events based on the amount of buried events, the amount of real events, and the amount of filtered events; for example, the server determines the buried event collection rate and the parameter collection rate based on the amount of buried events, the amount of real events, and the amount of filtered events, and the server evaluates whether the buried events are lost in the process of the software development kit processing the buried events through the event collection rate, and evaluates whether the parameters of the buried events are lost in the process of the software development kit processing the buried events through the parameter collection rate; for another example, after the software development kit releases a new version, business changes will not cause a large range of changes in the buried event collection rate and the parameter collection rate, while technical faults will cause a large range of changes in the buried event collection rate and the parameter collection rate, therefore, the influence of business changes can be excluded through the buried event collection rate and the parameter collection rate, and it can be accurately judged whether a technical fault occurs in the process of the software development kit processing the buried events; thereby improving the accuracy of evaluating the quality of the software development kit processing the buried events.

[0072] Referring to Figure 4 , Figure 4 A flowchart of an evaluation method provided by an embodiment of the present disclosure is shown, wherein the method can be executed by any electronic device, such as a server; as an optional implementation manner, the method can be executed by a server, for the convenience of description, in the description of some optional embodiments below, the server will be taken as an example to illustrate the execution subject of the method. As Figure 4 The evaluation method provided by the embodiment of the present disclosure includes the following steps: S401, receiving the amount of buried events, the amount of real events, and the amount of filtered events sent by the software development kit called by the application program in each terminal of the terminal set.

[0073] Specifically, the amount of buried events sent by the SDK called by the application program in each terminal is used to represent the total number of all buried events in the buried event set; the amount of real events is used to represent the number of buried events in the buried event set meeting the preset first rule, and the amount of filtered events is used to represent the number of buried events in the buried event set not meeting the preset first rule.

[0074] S402, determine the event collection rate and the parameter collection rate of the software development kit based on the event quantity, the real event quantity and the filtered event quantity sent by each terminal.

[0075] Specifically, the server groups each terminal based on the event quantity, the real event quantity and the filtered event quantity sent by each terminal, and determines the event collection rate corresponding to each group, i.e. the event collection rate of the software development kit; the server samples each terminal based on the event quantity, the real event quantity and the filtered event quantity sent by each terminal, and determines the parameter collection rate corresponding to each group after sampling, i.e. the parameter collection rate of the software development kit.

[0076] S403, determine the quality score of the software development kit in processing the event based on the event collection rate and the parameter collection rate of the software development kit.

[0077] Specifically, for example, if the event collection rate of the software development kit is less than a first preset threshold, or the parameter collection rate of the software development kit is less than a second preset threshold, it is determined that the quality score of the software development kit in processing the event is unqualified.

[0078] For another example, if the event collection rate of the software development kit is less than a third preset threshold, and the parameter collection rate of the software development kit is less than a fourth preset threshold, it is determined that the quality score of the software development kit in processing the event is unqualified.

[0079] In the embodiments of the present disclosure, the terminal determines the event quantity, the real event quantity and the filtered event quantity through the software development kit, and the server evaluates the quality of the software development kit in processing the event based on the event quantity, the real event quantity and the filtered event quantity; for example, the server determines the event collection rate and the parameter collection rate based on the event quantity, the real event quantity and the filtered event quantity, and evaluates whether the event is lost in the process of the software development kit processing the event through the event collection rate, and evaluates whether the parameter of the event is lost in the process of the software development kit processing the event through the parameter collection rate; for another example, after the software development kit releases a new version, business changes will not cause a large range of changes in the event collection rate and the parameter collection rate, while technical faults will cause a large range of changes in the event collection rate and the parameter collection rate, therefore, the influence of business changes can be excluded through the event collection rate and the parameter collection rate, and it can be accurately judged whether a technical fault occurs in the process of the software development kit processing the event; thereby the accuracy of evaluating the quality of the software development kit in processing the event is improved.

[0080] In an embodiment, based on the amount of the buried point events, the amount of real events and the amount of filtered events sent by each terminal, the buried point event collection rate and the parameter collection rate for the software development kit are determined, including: Grouping the terminals based on the parameters in the buried point events sent by each terminal, at least one first group is determined; Determining the buried point event collection rate corresponding to the terminals included in each first group in the at least one first group; Based on the buried point event collection rate corresponding to the terminals included in each first group in the at least one first group, the buried point event collection rate corresponding to each first group is determined, and the buried point event collection rate corresponding to each first group is determined as the buried point event collection rate for the software development kit; Based on the terminal identifier in the buried point events sent by each terminal, a plurality of terminals are determined from the terminals, and the plurality of terminals are grouped based on the parameters in the buried point events sent by the plurality of terminals, at least one second group is determined; Determining the parameter collection rate corresponding to the terminals included in each second group in the at least one second group; Based on the parameter collection rate corresponding to the terminals included in each second group in the at least one second group, the parameter collection rate corresponding to each second group is determined, and the parameter collection rate corresponding to each second group is determined as the parameter collection rate for the software development kit.

[0081] Specifically, for example, 1000 terminals send 1000 SLA buried point events to the server, and the server groups the terminals based on common parameters (such as business identifier, buried point event reporting date, APP version number, etc.) in the 1000 SLA buried point events, and terminals with the same common parameters are attributed to a group, for example, the 1000 terminals are divided into two groups, the two groups are first group A and first group B, first group A includes 3 terminals, and first group B includes 997 terminals, that is, the terminals are grouped based on the parameters in the buried point events sent by each terminal, and at least one first group is determined; the server determines the buried point event collection rate A, the buried point event collection rate B and the buried point event collection rate C corresponding to the 3 terminals in the first group A based on formula (1); the server calculates the average value between the buried point event collection rate A, the buried point event collection rate B and the buried point event collection rate C, and determines the average value as the buried point event collection rate corresponding to the first group A; based on the same calculation method, the server determines the buried point event collection rate corresponding to the first group B; the server evaluates whether the buried point event is lost in the process of processing the buried point event by the SDK through the buried point event collection rate corresponding to the first group A and the buried point event collection rate corresponding to the first group B; wherein the buried point event collection rate corresponding to the first group A and the buried point event collection rate corresponding to the first group B are the buried point event collection rate for the software development kit.

[0082] For example, the SDKs of 1000 terminals respectively send the common parameters in the buried point events corresponding to the target real event amount, the real event amount and the buried point event amount to the server. The server performs sampling according to a preset sampling rate of 1% based on the common parameters (for example, terminal identifiers) sent by the SDKs of the terminals, selects 10 terminals from the 1000 terminals as samples, that is, based on the terminal identifiers in the buried point events sent by the terminals, performs sampling processing, and determines a plurality of terminals (for example, 10 terminals) from the terminals (for example, 1000 terminals); the server performs grouping based on the common parameters (for example, business identifiers, buried point event reporting dates, APP version numbers, etc.) sent by the SDKs of the 10 terminals, attributes terminals with the same common parameters to a group, for example, divides the 10 terminals into two groups, the two groups being a second group A and a second group B, the second group A including 3 terminals and the second group B including 7 terminals, that is, based on the parameters in the buried point events sent by the plurality of terminals, groups the plurality of terminals, and determines at least one second group; the server determines the parameter collection rates A, B and C respectively corresponding to the 3 terminals in the second group A based on formula (2); the server calculates the average value between the parameter collection rates A, B and C, and determines the average value as the parameter collection rate corresponding to the second group A; based on the same calculation method, the server determines the parameter collection rate corresponding to the second group B; the server evaluates whether the parameters of the buried point events are lost in the process of processing the buried point events by the SDKs through the parameter collection rate corresponding to the second group A and the parameter collection rate corresponding to the second group B; wherein the parameter collection rate corresponding to the second group A and the parameter collection rate corresponding to the second group B are the parameter collection rates for the software development kit.

[0083] In one embodiment, based on the device identifiers in the buried point events sent by the terminals, sampling processing is performed to determine a plurality of terminals from the terminals, including: Based on the terminal identifiers in the buried point events sent by the terminals and a preset sampling rate, a plurality of terminals are determined from the terminals through a string hash algorithm.

[0084] Specifically, for example, the preset sampling rate is 1%, the server performs hash operation on the terminal identifier (terminal ID) through a string hash algorithm to obtain a hash; the string hash algorithm is shown in formula (3): hash=s[0]×31^(n-1)+s[1]×31^(n-2)+...+s[n-1] Formula (3) Wherein, s[0] represents the ASCII code value of the 1st character in the string, s[1] represents the ASCII code value of the 2nd character in the string, and so on, s[n-1] represents the ASCII code value of the n-1th character in the string, if the value of hash satisfies abs(hash%10000)<1, the terminal identification corresponding terminal sampling passes, that is, the sampling terminal is taken as a sample terminal, abs(hash%10000) represents the absolute value of the remainder obtained by dividing hash by 10000; each terminal in the plurality of terminals is a sample terminal.

[0085] By applying the embodiments of the present disclosure, at least the following beneficial effects are achieved: The server determines the burying point event collection rate and the parameter collection rate based on the burying point event quantity, the real event quantity and the filtered event quantity. The server evaluates whether the burying point event is lost in the process of the software development kit processing the burying point event through the event collection rate, and evaluates whether the parameter of the burying point event is lost in the process of the software development kit processing the burying point event through the parameter collection rate. After the software development kit releases a new version, the business change will not cause a large range change of the burying point event collection rate and the parameter collection rate, while the technical fault will cause a large range change of the burying point event collection rate and the parameter collection rate. Therefore, the influence of the business change can be excluded through the burying point event collection rate and the parameter collection rate, and it is accurately judged whether the technical fault occurs in the process of the software development kit processing the burying point event. Thus, the accuracy of the evaluation on the quality of the software development kit processing the burying point event is improved.

[0086] In order to better understand the method provided by the embodiments of the present disclosure, the scheme of the embodiments of the present disclosure will be further described below in combination with examples of specific application scenarios.

[0087] In one embodiment, as Figure 3As shown, the buried point event reporting process includes: the SDK starts collecting buried point events; the SDK calls the interface (buried point event collection interface) to count the buried point events, the buried point event count corresponds to the call interface count ID, and the buried point event count ID is saved in the cache; the SDK executes the buried point event processing process, that is, the process of the SDK processing the buried point event, which includes: the SDK determines whether the buried point event is compliant, that is, whether the buried point event meets the preset first rule, if the SDK determines that the buried point event meets the preset first rule, it is determined that the buried point event is a real event, and the real event is reported to the first backend server, and at the same time, the real event reporting count is performed, that is, the real event count, if the SDK determines that the buried point event does not meet the preset first rule, it is determined that the buried point event is a filtered event, and the filtered event is discarded, and at the same time, the filtered event count is performed, and the filtering reason corresponding to the filtered event is recorded; the SDK saves the real event count ID, the filtered event count ID, and the filtering reason of the filtered event to the cache.

[0088] It should be noted that the SDK internally counts from the start of the buried point event counting by the SDK calling the interface (buried point event collection interface) to the end of the real event counting; the possible logical end position in the buried point event reporting process is, for example, due to various filtering reasons, the buried point event (filtered event) is filtered and discarded, and at the same time, the filtered event count is performed, so that the process ends (return); the possible logical end position in the buried point event reporting process is, for example, the buried point event is determined to be a real event, and at the same time, the real event count is performed, so that the process ends (return).

[0089] As shown in Figure 3 The SDK triggers the SLA buried point event reporting process once a minute through the timer, which includes: the SDK calls the initial value and the terminal value corresponding to the buried point event count ID, the initial value and the terminal value corresponding to the real event count ID, the initial value and the terminal value corresponding to the filtered event count ID, and the filtering reason of the filtered event recorded in the cache; based on the initial value and the terminal value corresponding to the buried point event count ID, the initial value and the terminal value corresponding to the real event count ID, and the initial value and the terminal value corresponding to the filtered event count ID, the SDK respectively determines the amount of buried point events (the amount of call interfaces), the amount of real events, and the amount of filtered events, which are parameters of the SLA buried point event; the SDK constructs the SLA buried point event by using the amount of buried point events, the amount of real events, the amount of filtered events, the common parameters in the buried point events corresponding to the amount of buried point events, and the filtering reason of the filtered events corresponding to the amount of filtered events; the SDK sends the SLA buried point event to the server, which is the second backend server; the server determines the buried point event collection rate based on the service level agreement buried point event.

[0090] In one embodiment, in a process requiring a series of complex processing steps, if it is necessary to count the success rate of the entire process, the evaluation method provided by the embodiments of the present disclosure can be used. For example, in a factory pipeline scenario, the evaluation method provided by the embodiments of the present disclosure is used to count the amount of raw materials, the amount of waste products and the amount of finished products; wherein the amount of raw materials, the amount of waste products and the amount of finished products correspond to the amount of buried point events, the amount of filtered events and the amount of real events respectively.

[0091] In one specific application scenario embodiment, for example, an evaluation scenario, refer to Figure 5 , a processing flow of an evaluation method is shown, as Figure 5 shown, the processing flow of the evaluation method provided by the embodiments of the present disclosure includes the following steps: S501, the target terminal in the terminal set initializes the original software development kit to obtain the software development kit, and generates the terminal identification of the target terminal, the buried point event count identification, the real event count identification and the filtered event count identification.

[0092] Specifically, for example, the original SDK (original software development kit) in the terminal is initialized to obtain the SDK (software development kit), and the terminal ID (terminal identification of the terminal), the buried point event count ID (buried point event count identification), the real event count ID (real event count identification), the filtered event count ID (filtered event count identification) and the like of the terminal are generated.

[0093] S502, the target terminal calls the software development kit through the application program to obtain the buried point event set for the application program within a preset time period and the amount of buried point events corresponding to the buried point event set.

[0094] Specifically, the preset time period is, for example, 1 minute. For example, the software development kit called by the application program in the target terminal determines the first count value corresponding to the buried point event count identification and the second count value corresponding to the buried point event count identification, the first count value is used to represent the buried point event count at the first time point, the second count value is used to represent the buried point event count at the second time point, the first time point is the starting time point of the preset time period, and the second time point is the ending time point of the preset time period; based on the first count value and the second count value, the amount of buried point events corresponding to the buried point event set is determined.

[0095] S503, the target terminal determines the real event set and the filtered event set based on the buried point event set and a preset first rule through the software development kit.

[0096] Specifically, for example, as Figure 3As shown, the process of processing the buried point event (the process of processing the buried point event by the SDK) includes that the SDK judges whether the buried point event is compliant, that is, the SDK judges whether the buried point event meets the preset first rule; if the SDK determines that the buried point event meets the preset first rule, it is determined that the buried point event is a real event, and the real event is reported to the first backend server, and at the same time, the real event reporting count is performed, that is, the real event count; if the SDK determines that the buried point event does not meet the preset first rule, it is determined that the buried point event is a filtered event, and the filtered event is discarded, and at the same time, the filtered event count is performed.

[0097] S504, the target terminal performs counting processing on the real event set through the software development kit to determine the real event quantity, and performs counting processing on the filtered event set through the software development kit to determine the filtered event quantity.

[0098] Specifically, for example, the target terminal performs counting processing on the real event set through the software development kit to determine the third count value corresponding to the real event count identifier and the fourth count value corresponding to the real event count identifier, the third count value is used to represent the real event count at the first time point, and the fourth count value is used to represent the real event count at the second time point, the first time point is the starting time point of the preset time period, and the second time point is the ending time point of the preset time period; based on the third count value and the fourth count value, the real event quantity corresponding to the real event set is determined; the preset time period is, for example, 1 minute.

[0099] For example, the target terminal performs counting processing on the filtered event set through the software development kit to determine the fifth count value corresponding to the filtered event count identifier and the sixth count value corresponding to the filtered event count identifier, the fifth count value is used to represent the filtered event count at the first time point, and the sixth count value is used to represent the filtered event count at the second time point, the first time point is the starting time point of the preset time period, and the second time point is the ending time point of the preset time period; based on the fifth count value and the sixth count value, the filtered event quantity corresponding to the filtered event set is determined; the preset time period is, for example, 1 minute.

[0100] S505, the target terminal constructs a service level agreement buried point event through the software development kit, and sends the service level agreement buried point event to the server.

[0101] Specifically, the target terminal constructs a service level agreement buried point event through the software development kit by using the buried point event quantity, the real event quantity, the filtered event quantity, the parameters in the buried point event corresponding to the buried point event quantity, and the filtering reason of the filtered event corresponding to the filtered event quantity.

[0102] S506, the target terminal uses a software development kit to determine the parameter values ​​and target real event quantity that conform to the preset second rule based on the parameter values ​​of the parameters in each real event corresponding to the real event quantity and the preset second rule.

[0103] Specifically, the target real event quantity is used to characterize the number of real events corresponding to parameter values ​​that conform to the preset second rule.

[0104] S507, the target terminal sends the target actual event quantity, the actual event quantity, and the parameters in the event corresponding to the event tracking quantity to the server through the software development kit.

[0105] Specifically, the server is, for example, Figure 3 The second backend server is shown in the diagram.

[0106] S508: The server determines the data collection rate of the data collection event for the data collection event based on the service level protocol data collection event sent by the software development kits of each terminal in the terminal set, and determines the parameter collection rate for the data collection event based on the target real event volume, real event volume and data collection event volume sent by the data collection event for each terminal.

[0107] Specifically, the terminal set includes the target terminals. For example, among 1000 terminals, the SDK sends 1000 SLA event tracking points to the server. The server groups these 1000 SLA event tracking points based on common parameters (such as service identifier, event tracking reporting date, APP version number, etc.), and assigns terminals with the same common parameters to a group. For example, the 1000 terminals are divided into two groups, namely the first group A and the first group B. The first group A includes 3 terminals, and the first group B includes 997 terminals. That is, based on the parameters in the event tracking points sent by each terminal, each terminal is grouped to determine at least one first group. The server determines the event tracking points corresponding to the 3 terminals in the first group A based on formula (1). The server calculates the average of the three event collection rates: A, B, and C. This average is then used as the event collection rate for the first group A. Using the same calculation method, the server determines the event collection rate for the first group B. The server uses the event collection rates for the first group A and the first group B to evaluate whether event collection is lost during SDK event processing. The event collection rates for the first group A and the first group B are the event collection rates for the software development kit (SDK).

[0108] For example, the SDKs of 1000 terminals respectively send the common parameters in the corresponding buried point events of the target real event amount, the real event amount and the buried point event amount to the server. The server samples according to a preset sampling rate of 1% based on the common parameters (for example, terminal identifier) sent by the SDKs of the terminals, selects 10 terminals as samples from the 1000 terminals, that is, determines a plurality of terminals (for example, 10 terminals) from the terminals (for example, 1000 terminals) based on the terminal identifier in the buried point events sent by the terminals. The server groups based on the common parameters (for example, business identifier, buried point event reporting date, APP version number, etc.) sent by the SDKs of the 10 terminals, attributes terminals with the same common parameters to a group, for example, divides the 10 terminals into two groups, the two groups are a second group A and a second group B, the second group A includes 3 terminals, and the second group B includes 7 terminals, that is, groups the plurality of terminals based on the parameters in the buried point events sent by the plurality of terminals, and determines at least one second group. The server determines the parameter collection rate A, the parameter collection rate B and the parameter collection rate C respectively corresponding to the 3 terminals in the second group A based on formula (2). The server calculates the average value between the parameter collection rate A, the parameter collection rate B and the parameter collection rate C, and determines the average value as the parameter collection rate corresponding to the second group A. Based on the same calculation method, the server determines the parameter collection rate corresponding to the second group B. The server evaluates whether the parameters of the buried point events are lost in the process of the SDK processing the buried point events through the parameter collection rate corresponding to the second group A and the parameter collection rate corresponding to the second group B. The parameter collection rate corresponding to the second group A and the parameter collection rate corresponding to the second group B are the parameter collection rates for the software development kit.

[0109] S509, the server determines the quality score of the software development kit processing the buried point events based on the buried point event collection rate for the software development kit and the parameter collection rate.

[0110] Specifically, for example, if the buried point event collection rate for the software development kit is less than a first preset threshold, or the parameter collection rate for the software development kit is less than a second preset threshold, it is determined that the quality score of the software development kit processing the buried point events is unqualified.

[0111] For another example, if the buried point event collection rate for the software development kit is less than a third preset threshold, and the parameter collection rate for the software development kit is less than a fourth preset threshold, it is determined that the quality score of the software development kit processing the buried point events is unqualified.

[0112] It should be noted that in an ideal case, if the event buried point event collection rate is less than 100%, it indicates that the buried point events are lost due to unknown SDK internal reasons.

[0113] The embodiments of the present disclosure have at least the following beneficial effects: The embodiments of the present disclosure are independent of business logic, and can be applied to any business access. The SDK automatically collects the buried point events, and can be applied to the statistical buried point event collection rate and parameter collection rate provided by the embodiments of the present disclosure, regardless of the buried point events corresponding to the occasions such as APP startup, page exposure, element exposure, element click, or other buried point events. The embodiments of the present disclosure do not depend on the stability of the reporting channel (the reporting channel is the SDK reporting process, which may cause the SDK to lose some buried point events due to network interruption). The detection range of each SLA buried point event is self-closed loop. The buried point event collection rate of a corresponding terminal within 1 minute is detected. If the network condition causes the SDK to lose some SLA buried point events, the detection result of the event collection rate will not be affected in the case of a large sample. In combination with the event collection rate and the parameter collection rate, a quality detection system for the SDK can be formed, which is independent of the specific business scenario. Once the buried point events are lost or the parameters of the buried point events are lost due to technical failure, the problem can be quickly found through the event collection rate and the parameter collection rate, thereby improving the accuracy of evaluating the quality of the SDK in processing the buried point events.

[0114] The embodiments of the present disclosure also provide an evaluation device applied to a target terminal. A structure diagram of the evaluation device is shown in Figure 6 The evaluation device 60 includes a first processing module 601, a second processing module 602, and a third processing module 603.

[0115] The first processing module 601 is configured to acquire, by a software development kit called by an application program in the target terminal, a set of buried point events for the application program and a buried point event quantity corresponding to the set of buried point events within a preset time period. The buried point event quantity is used to represent the total number of all buried point events in the set of buried point events. The second processing module 602 is configured to determine, by the software development kit, a real event quantity and a filtered event quantity based on the set of buried point events. The real event quantity is used to represent the number of buried point events in the set of buried point events that meet a preset first rule. The filtered event quantity is used to represent the number of buried point events in the set of buried point events that do not meet the preset first rule. The third processing module 603 is configured to send, by the software development kit, the buried point event quantity, the real event quantity, and the filtered event quantity to a server, so that the server evaluates the quality of the software development kit in processing the buried point events based on the buried point event quantity, the real event quantity, and the filtered event quantity corresponding to each terminal in a terminal set. The terminal set includes the target terminal, and each terminal in the terminal set includes the software development kit.

[0116] In one embodiment, the first processing module 601 is further configured to: Initialize the original software development kit in the target terminal to obtain the software development kit, and generate a terminal identifier, a buried point event count identifier, a real event count identifier and a filtered event count identifier of the target terminal; The first processing module 601 is specifically configured to: Determine, by the software development kit called by the application program in the target terminal, a first count value corresponding to the buried point event count identifier and a second count value corresponding to the buried point event count identifier, the first count value being used to represent the buried point event count at a first time point, and the second count value being used to represent the buried point event count at a second time point, the first time point being a starting time point of a preset time period, and the second time point being an ending time point of the preset time period; Determine, based on the first count value and the second count value, a buried point event quantity corresponding to the buried point event set.

[0117] In one embodiment, the second processing module 602 is specifically configured to: Determine, based on the buried point event set and a preset first rule, a real event set and a filtered event set by the software development kit; Determine, by the software development kit, a real event quantity by counting the real event set, and determine, by the software development kit, a filtered event quantity by counting the filtered event set, each real event in the real event set being used to represent a buried point event that meets the preset first rule, and each filtered event quantity in the filtered event set being used to represent a buried point event that does not meet the preset first rule.

[0118] In one embodiment, the second processing module 602 is specifically configured to: Determine, by the software development kit, a third count value corresponding to the real event count identifier and a fourth count value corresponding to the real event count identifier by counting the real event set, the third count value being used to represent the real event count at the first time point, and the fourth count value being used to represent the real event count at the second time point, the first time point being the starting time point of the preset time period, and the second time point being the ending time point of the preset time period; Determine, based on the third count value and the fourth count value, a real event quantity corresponding to the real event set.

[0119] In one embodiment, the second processing module 602 is specifically configured to: Determine, by the software development kit, a fifth count value corresponding to the filtered event count identifier and a sixth count value corresponding to the filtered event count identifier by counting the filtered event set, the fifth count value being used to represent the filtered event count at the first time point, and the sixth count value being used to represent the filtered event count at the second time point, the first time point being the starting time point of the preset time period, and the second time point being the ending time point of the preset time period; Determine a filtered event amount corresponding to the filtered event set based on the fifth count value and the sixth count value.

[0120] In an embodiment, the third processing module 603 is further configured to: Determine a service level agreement (SLA) event based on the amount of the buried point event, the amount of the real event, the amount of the filtered event, parameters in the buried point event corresponding to the amount of the buried point event, and filtering reasons of the filtered event corresponding to the amount of the filtered event. Determine, by the software development kit (SDK), a parameter value meeting the preset second rule and a target real event amount based on parameter values of the parameters in the real event corresponding to the amount of the real event and the preset second rule, the target real event amount being used to represent a quantity of the real event corresponding to the parameter value meeting the preset second rule.

[0121] In an embodiment, the third processing module 603 is specifically configured to: Send, by the SDK, the SLA event to a server, so that the server determines a buried point event collection rate based on the SLA event. Send, by the SDK, the parameters in the buried point event corresponding to the target real event amount, the amount of the real event, and the amount of the buried point event to the server, so that the server determines a parameter collection rate based on the parameters in the buried point event corresponding to the target real event amount, the amount of the real event, and the amount of the buried point event.

[0122] By applying the embodiments of the present disclosure, at least the following beneficial effects are achieved: The SDK called by the application program in the target terminal is used to obtain a set of buried point events for the application program and a buried point event quantity corresponding to the set of buried point events within a preset time period, the buried point event quantity being used to represent a total number of all the buried point events in the set of buried point events; based on the set of buried point events, the SDK is used to determine a real event quantity and a filtered event quantity, the real event quantity being used to represent a number of the buried point events in the set of buried point events that meet a preset first rule, and the filtered event quantity being used to represent a number of the buried point events in the set of buried point events that do not meet the preset first rule; the SDK is used to send the buried point event quantity, the real event quantity, and the filtered event quantity to the server, so that the server evaluates the quality of the SDK in processing the buried point events based on the buried point event quantity, the real event quantity, and the filtered event quantity corresponding to each terminal in the set of terminals; in this way, the terminal determines the buried point event quantity, the real event quantity, and the filtered event quantity through the SDK, so that the server evaluates the quality of the SDK in processing the buried point events based on the buried point event quantity, the real event quantity, and the filtered event quantity; for example, the server determines a buried point event collection rate and a parameter collection rate based on the buried point event quantity, the real event quantity, and the filtered event quantity, and evaluates whether the buried point events are lost or the parameters of the buried point events are lost in the process of the SDK processing the buried point events through the event collection rate and the parameter collection rate; for another example, after the SDK releases a new version, business changes do not cause a large range of changes in the buried point event collection rate and the parameter collection rate, while technical faults cause a large range of changes in the buried point event collection rate and the parameter collection rate, so that the influence of the business changes can be excluded through the buried point event collection rate and the parameter collection rate, and it can be accurately judged whether a technical fault occurs in the process of the SDK processing the buried point events; thereby the accuracy of evaluating the quality of the SDK in processing the buried point events is improved.

[0123] The embodiments of the present disclosure further provide an evaluation device applied to a server, and a structure diagram of the evaluation device is shown in Figure 7 The evaluation device 70 comprises a fourth processing module 701, a fifth processing module 702, and a sixth processing module 703.

[0124] The fourth processing module 701 is configured to receive the buried point event quantity, the real event quantity, and the filtered event quantity sent by the SDK called by the application program in each terminal in the set of terminals. The fifth processing module 702 is configured to determine a buried point event collection rate and a parameter collection rate for the SDK based on the buried point event quantity, the real event quantity, and the filtered event quantity sent by each terminal. The sixth processing module 703 is configured to determine a quality score of the SDK in processing the buried point events based on the buried point event collection rate and the parameter collection rate for the SDK.

[0125] In an embodiment, the sixth processing module 703 is specifically configured to: group the terminals based on the parameters in the buried point events sent by the terminals, and determine at least one first group; determine a buried point event collection rate corresponding to the terminals included in each of the at least one first group; determine a buried point event collection rate corresponding to each of the at least one first group based on the buried point event collection rate corresponding to the terminals included in each of the at least one first group, and determine the buried point event collection rate corresponding to each of the at least one first group as a buried point event collection rate for the software development kit; determine a plurality of terminals from the terminals based on the terminal identifiers in the buried point events sent by the terminals, and group the plurality of terminals based on the parameters in the buried point events sent by the plurality of terminals, and determine at least one second group; determine a parameter collection rate corresponding to the terminals included in each of the at least one second group; determine a parameter collection rate corresponding to each of the at least one second group based on the parameter collection rate corresponding to the terminals included in each of the at least one second group, and determine the parameter collection rate corresponding to each of the at least one second group as a parameter collection rate for the software development kit.

[0126] In an embodiment, the sixth processing module 703 is specifically configured to: determine a plurality of terminals from the terminals based on the terminal identifiers in the buried point events sent by the terminals and a preset sampling rate, by using a string hash algorithm.

[0127] According to the embodiments of the present disclosure, at least the following beneficial effects are achieved: The terminal uses a Software Development Kit (SDK) to determine the number of tracked events, actual events, and filtered events. The server then evaluates the quality of the SSD's handling of tracked events based on these metrics. For example, the server determines the tracked event collection rate and parameter collection rate. The server uses the event collection rate to assess whether tracked events are lost during SSD processing, and the parameter collection rate to assess whether parameters are lost. Furthermore, while business changes after a new SSD release may not cause significant fluctuations in the tracked event collection rate and parameter collection rate, technical malfunctions will. Therefore, the tracked event collection rate and parameter collection rate can eliminate the impact of business changes and accurately determine whether technical malfunctions have occurred during SSD processing, thus improving the accuracy of the evaluation of the SSD's handling of tracked events.

[0128] This disclosure also provides an electronic device, the structural schematic diagram of which is shown below. Figure 8 As shown, Figure 8 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this disclosure.

[0129] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with this disclosure. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0130] The bus 4002 can include a path that transmits information between the above components. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, and the like. For convenience of representation, Figure 8 Only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0131] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magneto-optical disk storage (including a compact disk, a laser disk, an optical disk, a digital versatile disk, a Blu-ray disk, and the like), a magnetic disk storage medium, other magnetic storage device, or any other medium that can be used to carry or store computer programs and that can be read by a computer, without limitation.

[0132] The memory 4003 is used to store a computer program for implementing the embodiments of the present disclosure, and is controlled by the processor 4001 to execute. The processor 4001 is used to execute the computer program stored in the memory 4003 to realize the steps shown in the foregoing method embodiments.

[0133] The electronic device includes, but is not limited to, a terminal, a server, and the like.

[0134] By applying the embodiments of the present disclosure, at least the following beneficial effects are achieved: The terminal determines the amount of the buried point event, the amount of the real event and the amount of the filtered event through the software development kit, and the server evaluates the quality of the software development kit processing the buried point event based on the amount of the buried point event, the amount of the real event and the amount of the filtered event; for example, the server determines the buried point event collection rate and the parameter collection rate based on the amount of the buried point event, the amount of the real event and the amount of the filtered event, and the server evaluates whether the buried point event is lost in the process of the software development kit processing the buried point event through the event collection rate, and evaluates whether the parameter of the buried point event is lost in the process of the software development kit processing the buried point event through the parameter collection rate; for another example, after the software development kit releases a new version, business changes will not cause a large range of changes in the buried point event collection rate and the parameter collection rate, and technical faults will cause a large range of changes in the buried point event collection rate and the parameter collection rate, therefore, the influence of business changes can be excluded through the buried point event collection rate and the parameter collection rate, and whether a technical fault occurs in the process of the software development kit processing the buried point event can be accurately judged; thereby the accuracy of evaluating the quality of the software development kit processing the buried point event is improved.

[0135] The embodiment of the present disclosure provides a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps and corresponding contents of the foregoing method embodiments.

[0136] The embodiment of the present disclosure also provides a computer program product, which comprises a computer program. The computer program is executed by a processor to implement the steps and corresponding contents of the foregoing method embodiments.

[0137] It should be understood that, although the flowcharts of the embodiments of the present disclosure indicate the respective operation steps by arrows, the implementation order of the steps is not limited to the order indicated by the arrows. Unless otherwise specified herein, in some implementation scenarios of the embodiments of the present disclosure, the implementation steps in each flowchart can be executed in other orders as required. In addition, part or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on the actual implementation scenario. Part or all of the sub-steps or stages can be executed at the same time, and each of the sub-steps or stages can also be executed at different times. In the scenario where the execution times are different, the execution order of the sub-steps or stages can be flexibly configured as required, and the embodiments of the present disclosure do not limit this.

[0138] The above is only an optional implementation manner of some implementation scenarios of the present disclosure, and it should be pointed out that, for ordinary technical personnel in the technical field, other similar implementation manners based on the technical idea of the present disclosure without departing from the technical concept of the present disclosure also belong to the protection scope of the embodiments of the present disclosure.

Claims

1. A method of evaluation, performed by a target terminal, characterized by, The method comprises the following steps: obtaining, by a software development kit called by an application program in the target terminal, a set of buried point events of the application program in a preset time period and a buried point event quantity corresponding to the set of buried point events, the buried point event quantity being used to represent the total number of all buried point events in the set of buried point events; determining, by the software development kit based on the set of buried point events, a real event quantity and a filtered event quantity, the real event quantity being used to represent the number of buried point events in the set of buried point events that meet a preset first rule, and the filtered event quantity being used to represent the number of buried point events in the set of buried point events that do not meet the preset first rule; sending, by the software development kit, the buried point event quantity, the real event quantity and the filtered event quantity to a server, so that the server evaluates the quality of the software development kit in processing buried point events based on the buried point event quantity, the real event quantity and the filtered event quantity corresponding to each terminal in a terminal set, the terminal set including the target terminal and each terminal in the terminal set including the software development kit.

2. The method of claim 1, wherein, Before the step of obtaining, by a software development kit called by an application program in the target terminal, a set of buried point events of the application program in a preset time period and a buried point event quantity corresponding to the set of buried point events, the method further comprises the following steps: initializing an original software development kit in the target terminal to obtain a software development kit, and generating a terminal identifier, a buried point event count identifier, a real event count identifier and a filtered event count identifier of the target terminal; the step of obtaining, by the software development kit called by the application program in the target terminal, the buried point event quantity corresponding to the set of buried point events comprises the following steps: determining, by the software development kit called by the application program in the target terminal, a first count value corresponding to the buried point event count identifier and a second count value corresponding to the buried point event count identifier, the first count value being used to represent the buried point event count at a first time point, and the second count value being used to represent the buried point event count at a second time point, the first time point being the starting time point of the preset time period, and the second time point being the ending time point of the preset time period; determining the buried point event quantity corresponding to the set of buried point events based on the first count value and the second count value.

3. The method of claim 1, wherein, the step of determining, by the software development kit based on the set of buried point events, a real event quantity and a filtered event quantity comprises the following steps: determining, by the software development kit based on the set of buried point events and the preset first rule, a set of real events and a set of filtered events; determining, by the software development kit, the real event quantity by counting the set of real events, and determining, by the software development kit, the filtered event quantity by counting the set of filtered events, each real event in the set of real events being used to represent a buried point event meeting the preset first rule, and each filtered event in the set of filtered events being used to represent a buried point event not meeting the preset first rule.

4. The method of claim 3, wherein, The counting processing of the software development kit for the real event set determines the real event amount, comprising: The counting processing of the software development kit for the real event set determines the real event count identifier corresponding third count value and the real event count identifier corresponding fourth count value, the third count value is used to represent the real event count of the first time point, and the fourth count value is used to represent the real event count of the second time point, the first time point is the starting time point of the preset time period, and the second time point is the ending time point of the preset time period; Based on the third count value and the fourth count value, the real event amount corresponding to the real event set is determined.

5. The method of claim 3, wherein, The counting processing of the software development kit for the filtered event set determines the filtered event amount, comprising: The counting processing of the software development kit for the filtered event set determines the filtered event count identifier corresponding fifth count value and the filtered event count identifier corresponding sixth count value, the fifth count value is used to represent the filtered event count of the first time point, and the sixth count value is used to represent the filtered event count of the second time point, the first time point is the starting time point of the preset time period, and the second time point is the ending time point of the preset time period; Based on the fifth count value and the sixth count value, the filtered event amount corresponding to the filtered event set is determined.

6. The method of claim 1, wherein, Before the software development kit sends the buried point event amount, the real event amount and the filtered event amount to the server, it further comprises: Based on the buried point event amount, the real event amount, the filtered event amount, the parameter in the buried point event corresponding to the buried point event amount and the filtering reason of the filtered event corresponding to the filtered event amount, the service level agreement buried point event is determined; Based on the parameter value of the parameter in each real event corresponding to the real event amount and the preset second rule, the software development kit determines the parameter value meeting the preset second rule and the target real event amount, the target real event amount is used to represent the number of real events corresponding to the parameter value meeting the preset second rule.

7. The method of claim 6, wherein, The software development kit sends the buried point event amount, the real event amount and the filtered event amount to the server, comprising: The software development kit sends the service level agreement buried point event to the server, so that the server determines the buried point event collection rate based on the service level agreement buried point event; The software development kit sends the target real event amount, the real event amount and the parameter in the buried point event corresponding to the buried point event amount to the server, so that the server determines the parameter collection rate based on the target real event amount, the real event amount and the parameter in the buried point event corresponding to the buried point event amount.

8. An evaluation method executed by a server, characterized by, Comprise: The application program in each terminal of the terminal set receives the buried point event amount, the real event amount and the filtered event amount sent by the software development kit; determine a quality score of the software development kit in handling the trace events based on the trace event collection rate and the parameter collection rate for the software development kit. determine a quality score of the software development kit in handling the trace events based on the trace event collection rate and the parameter collection rate for the software development kit.

9. The method of claim 8, wherein, The determining of the trace event collection rate and the parameter collection rate for the software development kit based on the trace event quantity, the real event quantity and the filtered event quantity sent by each terminal comprises: group the terminals based on the parameters in the trace events sent by the terminals, and determine at least one first group; determine a trace event collection rate corresponding to each terminal included in the at least one first group; determine a trace event collection rate corresponding to each first group based on the trace event collection rate corresponding to the terminal included in each first group in the at least one first group, and determine the trace event collection rate corresponding to each first group as the trace event collection rate for the software development kit; determine a plurality of terminals from the terminals based on the terminal identifiers in the trace events sent by the terminals, and group the plurality of terminals based on the parameters in the trace events sent by the plurality of terminals, and determine at least one second group; determine a parameter collection rate corresponding to each terminal included in the at least one second group; determine a parameter collection rate corresponding to each second group based on the parameter collection rate corresponding to the terminal included in each second group in the at least one second group, and determine the parameter collection rate corresponding to each second group as the parameter collection rate for the software development kit.

10. The method of claim 9, wherein, The determining of the plurality of terminals from the terminals based on the device identifiers in the trace events sent by the terminals comprises: determine a plurality of terminals from the terminals based on the terminal identifiers in the trace events sent by the terminals and a preset sampling rate, by using a string hash algorithm.

11. An evaluation device applied to a target terminal, characterized in that, comprise: a first processing module, configured to acquire, by a software development kit called by an application program in the target terminal, a trace event set for the application program in a preset time period and a trace event quantity corresponding to the trace event set, the trace event quantity being used to represent a total number of all trace events in the trace event set; a second processing module, configured to determine, by the software development kit based on the trace event set, a real event quantity and a filtered event quantity, the real event quantity being used to represent a number of trace events in the trace event set that meet a preset first rule, and the filtered event quantity being used to represent a number of trace events in the trace event set that do not meet the preset first rule; The third processing module is configured to send the buried point event quantity, the real event quantity and the filtered event quantity to a server through the software development kit, so that the server evaluates the quality of the software development kit in processing the buried point event based on the buried point event quantity, the real event quantity and the filtered event quantity corresponding to each terminal in a terminal set, the terminal set including the target terminal, and each terminal in the terminal set including the software development kit.

12. An evaluation device applied to a server, characterized in that, The application comprises: The fourth processing module is configured to receive the buried point event quantity, the real event quantity and the filtered event quantity sent by the software development kit called by an application program in each terminal in a terminal set; The fifth processing module is configured to determine a buried point event collection rate and a parameter collection rate for the software development kit based on the buried point event quantity, the real event quantity and the filtered event quantity sent by each terminal; The sixth processing module is configured to determine a quality score of the software development kit in processing the buried point event based on the buried point event collection rate and the parameter collection rate for the software development kit.

13. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, is arranged to perform the method of any one of claims 1 to 12. The processor executes the computer program to implement the steps of the method in any one of claims 1-10.

14. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1-10.

15. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1-10.

Images