Software tracking quality assessment method, device, computing device and storage medium

Abstract

The embodiments of the present invention relate to the field of computer technology, and disclose a method, apparatus, computing device, and storage medium for assessing the quality of software tracking points. The method comprises: obtaining first data, wherein the first data comprises operation events, operation event pass rates, tracking points, and tracking point pass rates; inputting the first data into a tracking point quality assessment model, wherein the tracking point quality assessment model is trained based on sample data, wherein the sample data comprises sample operation events, sample operation event pass rates, sample tracking points, sample tracking point pass rates, and sample event quality judgment results, wherein the sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standard; obtaining the event quality judgment results output by the tracking point quality assessment model, wherein the event quality judgment results are used to indicate whether the software to be tested corresponding to the first data meets the online standard. The embodiments of the present invention improve testing efficiency.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and specifically to a method, apparatus, computing device, and storage medium for evaluating the quality of software tracking points. Background Art

[0002] In the field of internet technology, product and operations personnel need to understand user usage of certain functional modules and scenarios within client products through tracking and big data technology to monitor user groups, business, or activity status. This is achieved by embedding tracking triggers within code blocks and reporting the collected data to designated backend servers for aggregation and statistics. The quality of tracking data directly impacts the accuracy of subsequent reports, so it is essential to test the accuracy of tracking. Improving the efficiency of tracking testing is a pressing issue. Summary of the Invention

[0003] In view of the above problems, an embodiment of the present invention provides a method, apparatus, computing device and storage medium for evaluating the quality of software tracking, which are used to solve the problem of low efficiency of tracking testing.

[0004] According to a first aspect of an embodiment of the present invention, a method for evaluating the quality of tracking points in software is provided, the method comprising:

[0005] Acquire first data, where the first data includes operation events, operation event pass rates, tracking points, and tracking point pass rates;

[0006] Inputting the first data into a tracking quality assessment model, wherein the tracking quality assessment model is trained based on sample data, wherein the sample data includes sample operation events, sample operation event pass rates, sample tracking points, sample tracking pass rates, and sample event quality judgment results, wherein the sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standard;

[0007] Obtain an event quality judgment result output by the tracking quality assessment model, wherein the event quality judgment result is used to indicate whether the software under test corresponding to the first data meets the online standard.

[0008] In an optional manner, the tracking quality assessment model is trained in the following manner:

[0009] obtaining the sample data;

[0010] Input the sample operation events, sample operation event pass rates, sample buried points, and sample buried point pass rates in the sample data into a buried point quality assessment model to obtain a training result, wherein the training result is a training quality judgment result of the sample operation events;

[0011] According to the training results and the sample event quality judgment results in the sample data, the parameters of the embedding quality assessment model are adjusted until the error between the training results and the event quality judgment results in the sample data is less than or equal to the preset error, thereby obtaining a trained embedding quality assessment model.

[0012] In an optional manner, the sample data includes local operation data obtained by the tester through user interface UI testing of the software, and reported data reported by the software through the tracking software development kit SDK.

[0013] In an optional manner, the first data includes the behavior data of grayscale users in a grayscale verification environment and the grayscale reporting data reported by the software through the tracking software development kit SDK in the grayscale verification environment.

[0014] In an optional manner, the method further includes:

[0015] Performing consistency verification on the event quantity and event category of the local operation data and the reported data to obtain a first result;

[0016] Verifying the correctness of the sample embedding points in each event of the reported data to obtain a second result;

[0017] Calculate the sample embedding pass rate and the sample operation event pass rate based on the first result and the second result;

[0018] or,

[0019] The method further comprises:

[0020] Performing consistency verification on the grayscale user's behavior data and the event quantity and event category of the grayscale reporting data to obtain a third result;

[0021] Verify the correctness of the sample embedding points in each event of the grayscale reporting data to obtain a fourth result;

[0022] The point embedding pass rate and the operation event pass rate are calculated based on the third result and the fourth result.

[0023] In an optional manner, the sample embedding pass rate is calculated using the following formula:

[0024]

[0025] Among them, pr1 is the sample embedding pass rate; N correct1is the number of sample buried points that have passed. If the first result is that the consistency verification of the number of events and event categories of the local operation data and the reported data has passed, and the second result is that the correctness verification of the sample buried points in each event of the reported data has passed, then the sample buried points are passed. total1 The total number of reported sample burial points;

[0026] The embedding pass rate is calculated by the following formula:

[0027]

[0028] Among them, pr2 is the passing rate of buried points; N correct2 is the number of buried points that passed. If the third result is that the consistency verification of the number of events and event categories of the grayscale user's behavior data and the grayscale reporting data is passed, and the fourth result is that the correctness verification of the buried points in each event of the grayscale reporting data is passed, then the buried points are passed; N total2 The total number of reported burial points;

[0029] The pass rate of the sample operation event is calculated using the following formula:

[0030]

[0031] Where M is the total number of sample embedding points included in the sample operation event; PR3 is the pass rate of the sample operation event; p i is the passing rate of each sample embedding point corresponding to the sample operation event; θ i The weight of each sample's tracking pass rate as a percentage of the total sample event verification;

[0032] The operation event pass rate is calculated using the following formula:

[0033]

[0034] Where N is the total number of tracking points included in the operation event; PR4 is the pass rate of the operation event; p j is the pass rate of each buried point corresponding to the operation event; θ j The weight of each tracking point's pass rate as a percentage of the total event verification.

[0035] According to a second aspect of an embodiment of the present invention, a method for training a tracking quality assessment model is provided, the method comprising:

[0036] Obtaining sample data, the sample data including sample operation events, sample operation event pass rates, sample embedding points, sample embedding pass rates, and sample event quality judgment results, wherein the sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standard;

[0037] Input the sample operation events, sample operation event pass rates, sample buried points, and sample buried point pass rates in the sample data into a buried point quality assessment model to obtain a training result, wherein the training result is a training quality judgment result of the sample operation events;

[0038] According to the training results and the sample event quality judgment results in the sample data, the parameters of the embedding quality assessment model are adjusted until the error between the training results and the event quality judgment results in the sample data is less than or equal to the preset error, thereby obtaining a trained embedding quality assessment model.

[0039] According to a third aspect of an embodiment of the present invention, a device for evaluating the quality of tracking points of software is provided, the device comprising:

[0040] An acquisition module, configured to acquire first data, the first data including operation events, operation event pass rates, buried points, and buried point pass rates;

[0041] An input module, configured to input the first data into a tracking quality assessment model, wherein the tracking quality assessment model is trained based on sample data, wherein the sample data includes sample operation events, sample operation event pass rates, sample tracking points, sample tracking point pass rates, and sample event quality judgment results, wherein the sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standard;

[0042] An obtaining module is used to obtain the event quality judgment result output by the tracking quality assessment model, wherein the event quality judgment result is used to indicate whether the software under test corresponding to the first data meets the online standard.

[0043] According to a fourth aspect of an embodiment of the present invention, there is provided a computing device, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus;

[0044] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the method described above.

[0045] According to a fifth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores at least one executable instruction. When the executable instruction is executed on a computing device, the computing device executes the operation of the method described above.

[0046] The embodiment of the present invention inputs data such as operation events, operation event pass rates, tracking points and tracking point pass rates into a tracking point quality assessment model, and the tracking point quality assessment model automatically outputs event quality judgment results, indicating whether the software to be tested meets the online standards, thereby realizing automated tracking point testing, reducing the test personnel's review work on tracking point detection reports, and improving testing efficiency.

[0047] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to more clearly understand the technical means of the embodiments of the present invention, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the drawings. In the drawings:

[0049] Figure 1 A schematic diagram of an application scenario of an embodiment of the present invention is shown;

[0050] Figure 2 A flowchart of a method for evaluating the quality of software tracking provided by an embodiment of the present invention is shown;

[0051] Figure 3 A schematic block diagram of a process for obtaining first data in an embodiment of the present invention is shown;

[0052] Figure 4 Another schematic diagram of the process of evaluating the quality of software tracking points provided by an embodiment of the present invention is shown;

[0053] Figure 5 A functional module diagram of a system for configuring buried data inspection items according to an embodiment of the present invention is shown;

[0054] Figure 6 A flowchart of obtaining first data under a test environment according to an embodiment of the present invention is shown;

[0055] Figure 7 A flowchart of obtaining first data in a grayscale verification environment according to an embodiment of the present invention is shown;

[0056] Figure 8a A detailed flow chart of step a3 is shown;

[0057] Figure 8b A detailed flow chart of step b3 is shown;

[0058] Figure 9 A flowchart of a method for training a tracking quality assessment model provided by an embodiment of the present invention is shown;

[0059] Figure 10 A schematic diagram showing the structure of a software tracking quality assessment device provided by an embodiment of the present invention is shown;

[0060] Figure 11 A schematic structural diagram of a computing device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0061] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0062] Test tracking points are included in user operation events, and operation events are generally distinguished by calibrating keywords, setting categories, etc., and finally a report is generated through big data technology, which is fed back to the operator on the reporting status of the corresponding tracking points. In the testing phase, operation events include clicks, page exposure, playback, exit, etc. Under each operation event, the specific tracking content of the business personnel can be included. For example, the playback event contains specific information such as program number, loading delay, playback address, etc. Therefore, in the relevant technology, when the tester verifies the correctness of the tracking point, he first finds the operation event and then checks the specific information of the tracking point. In order to ensure the test efficiency of the tester, the test data will be directly stored in the log storage system, and then the log will be fed back to the tester through query, without the need for time-consuming operations such as generating reports.

[0063] Tracking verification is a crucial testing step before the launch of software (such as client apps). Conventional technologies only report the pass rate of test results, requiring testers to manually review tracking test reports and determine whether they meet launch requirements. During periods of frequent development and testing, the repeated verification workload for testers is enormous, resulting in low testing efficiency and difficulty maintaining quality.

[0064] Therefore, in response to the above problems, embodiments of the present invention provide a method, apparatus, computing device, and computer-readable storage medium for evaluating the quality of software tracking.

[0065] In order to better understand the solution of the embodiment of the present invention, Figure 1 The possible application scenarios of the embodiments of the present invention are briefly introduced.

[0066] Figure 1 In the example, the software to be tested is installed on the local terminal 101. Figure 1The computing device 102 (e.g., a server) in the embodiment of the present invention is capable of executing the tracking quality assessment method for software. In a test environment, the tester uploads local operation data to the computing device 102. The software under test in the terminal 101 calls the data provided by the tracking software development kit (SDK) and reports it to the computing device 102. The computing device 102 obtains the event quality judgment result based on the local operation data and the reported data, that is, the judgment result on whether the software under test meets the online standard.

[0067] Tracking SDK is a functional module for reporting user behavior data in APP products. APP developers report the data to be collected to the backend (for example, Figure 1 General user behavior data, such as user app launch events, app homepage exposure events, user playback events, and other behavioral data, can be reported to the backend to understand user behavior preferences and provide targeted services to improve user experience.

[0068] Figure 2 A flowchart of a method for evaluating the quality of software tracking provided by an embodiment of the present invention is shown. The method can be executed in a computing device such as a computer or server that provides software tracking quality evaluation for each client. For example, Figure 1 Executed by the computing device in .

[0069] Figure 3 A schematic block diagram of the process of obtaining first data in an embodiment of the present invention is shown. Figure 4 Another flow chart of the method for evaluating the quality of software tracking provided by an embodiment of the present invention is shown. Figure 3 As shown, the tracking data check item configuration module is used to register the tracking event list and the tracking verification rules for the data items in each event. The tracking verification module can read the information configured by the tracking data check item configuration module and check whether the received tracking data meets the requirements. Figure 5 FIG shows a functional module diagram of a system for configuring buried data inspection items in an embodiment of the present invention. Figure 5 As shown, the buried data check item configuration system includes a buried data check item configuration module and a database. The buried data check item configuration module includes:

[0070] The event configuration submodule is used to receive the tester's configuration information for new events, as well as sample metadata of correct data, such as configuring the launch event type: launcher, the exposure event type: expose, etc.

[0071] The rule configuration submodule is used to configure the tracking point verification rules corresponding to each reported event, including existence verification, range verification, enumeration verification, fuzzy matching and other functions. Existence verification is used to verify whether the tracking point exists, range verification is used to verify whether the length range of the tracking point meets the preset range, whether the numerical range of the tracking point meets the preset range, enumeration verification is used to verify the enumeration value of the reporting information field contained in the event, and fuzzy matching is used to verify whether the field contains a specific character or a certain format requirement. For example, for an exposure event, the event (type: expose) contains multiple reporting information fields, such as the exposure element name element, which cannot be empty, the exposure position position, which cannot be empty, and the operating system os, which is an enumeration value, which can be ios or android; and

[0072] The event and rule storage submodule is used to connect to the data storage module, such as MongoDB, MySQL and other databases, to respond to the testers' addition, modification, deletion and query operations on the tracking verification rules.

[0073] Table 1 shows examples of inspection items:

[0074] Table 1 Examples of inspection items

[0075] type Field 1 Field 2 Field 3 launcher key1 key2 key3 expose key4 key5 Key6 …… …… …… …… Type-n keyx Key keyz

[0076] like Figure 4As shown, after the tester configures the tracking verification rules on the tracking data check item configuration module, the UI test is started through the user interface testing (UI test) module to simulate user behavior, and the local operation data is saved locally through the local operation data storage module for subsequent test comparison. Then, the software to be tested reports the tracking data to the server through the tracking SDK. After the data is cleaned by the data cleaning module, the event classification module classifies the tracking data according to the event. The tester uploads the local operation data to the tracking verification module. At this point, the tracking verification module obtains two copies of data, one from the tester and the other from the software to be tested. Through the comparison and verification of the two copies of data, the tracking verification module can output the tracking detection report. The tracking verification module mainly completes the consistency verification and correctness verification of the tracking data. Among them, the consistency verification submodule receives the local operation data of the UI test and the tracking reporting data of the software to be tested, and is used to output the consistency results of the event, such as the number of reported events, the consistency of categories, etc. The correctness verification submodule uses the tracking verification rules configured by the tester for the event to verify whether the tracking in each operation event passes. The tracking verification rules include whether the tracking exists, whether the length range of the tracking meets the preset range, whether the numerical range of the tracking meets the preset range, fuzzy matching (whether the field contains a specific character or a certain format requirement), etc. The result summary submodule is used to count the pass rate of operation events and the tracking pass rate as the basis for the output of the tracking detection report. The tracking detection report output module outputs the tracking detection report. The tracking quality assessment module evaluates the tracking quality through the tracking quality assessment model.

[0077] like Figure 2 As shown, the method includes steps 21 to 23:

[0078] Step 21: Acquire first data, where the first data includes operation events, operation event pass rates, buried points, and buried point pass rates.

[0079] Among them, the first data includes local operation data obtained by the tester through the user interface UI testing of the software, and the reported data reported by the software through the tracking software development tool kit SDK.

[0080] Figure 6 FIG1 shows a flow chart of obtaining first data under a test environment according to an embodiment of the present invention. Figure 3 、 Figure 4 and Figure 6 , in a test environment, obtaining the first data further includes:

[0081] Step a1: receiving local operation data in a test environment, wherein the local operation data is data obtained by a tester through performing a user interface UI test on the software;

[0082] Step a2: receiving reported data, wherein the reported data is the tracking data reported by the software through the tracking software development kit SDK;

[0083] Step a3: Compare and verify the local operation data and the reported data to obtain the operation event pass rate and the tracking point pass rate.

[0084] Therefore, in a test environment, testers can upload local operation data to the tracking verification module, compare and verify it with the reported data reported by the tracking SDK, and verify it with the pre-registered tracking verification rules.

[0085] The tracking testers are unable to simulate the user's operation behavior in the real environment. Therefore, only obtaining the first data in the test environment will lead to incomplete test cases and failure to fully reflect the tracking problems. The grayscale verification environment is an environment in which a small range of experience users are selected to verify the product functions before the software product is released to the application market on a large scale. Once a product function is found to have huge defects, it will not be released to the application market on a large scale. In the grayscale verification environment, if it is found that the tracking data does not meet the requirements, it can be requested that the software to be tested cannot be released. In some embodiments, the first data may also include the behavior data of grayscale users in the grayscale verification environment and the grayscale reporting data reported by the software through the tracking software development kit SDK in the grayscale verification environment. Figure 7 The flowchart of obtaining the first data under the grayscale verification environment of the embodiment of the present invention is shown. Figure 3 、 Figure 4 and Figure 7 In a grayscale verification environment, obtaining sample data further includes:

[0086] Step b1: In the grayscale verification environment, receive grayscale user behavior data;

[0087] Step b2: receiving grayscale reporting data, wherein the grayscale reporting data is the tracking data reported by the software through the tracking software development kit SDK;

[0088] Step b3: Compare and verify the grayscale user's behavior data and the grayscale reporting data to obtain the operation event pass rate and the tracking point pass rate.

[0089] Therefore, it is possible to detect and verify the real behavior of the experiencing users in a grayscale verification environment, avoiding scenarios that cannot be covered by test cases and ensuring that the delivered software has a high tracking quality. Figure 8aDetailed flow chart of step a3 is shown. Figure 8a As shown, the comparison and verification of the local operation data and the reported data to obtain the operation event pass rate and the tracking pass rate (step a3) further includes:

[0090] Step c1: verifying the consistency of the event quantity and event category of the local operation data and the reported data to obtain a fifth result;

[0091] Step c2: verifying the correctness of the buried points in each event of the reported data to obtain a sixth result;

[0092] Specifically, correctness verification can be performed based on the tracking verification rules described above.

[0093] Step c3: Calculate the tracking pass rate and the operation event pass rate based on the fifth result and the sixth result.

[0094] The passing rate of the buried point is calculated by the following formula:

[0095]

[0096] Among them, pr2 is the passing rate of buried points; N correct2 is the number of buried points that passed. If the fifth result is that the consistency verification of the number of events and event categories of the local operation data and the reported data is passed, and the sixth result is that the correctness verification of the buried points in each event of the reported data is passed, then the buried points are passed; N total2 The total number of reported burial points;

[0097] The tracking pass rate is for a single checkpoint (the tracking point to be checked), for example, a specific field. For example, an exposure event has multiple fields: pageID, userID, from, uploadTime, etc. If a total of 1000 exposure data items are received, of which 900 have an empty pageID field and 800 have an empty from field, the corresponding pass rate for pageID is 90%, and the pass rate for the from field is 80%.

[0098] The operation event pass rate is calculated using the following formula:

[0099]

[0100] Where N is the total number of tracking points included in the operation event; PR4 is the pass rate of the operation event; p j is the pass rate of each buried point corresponding to the operation event; θ j The weight of each tracking point's pass rate as a percentage of the total event verification.

[0101] Specifically, the event pass rate refers to the pass rate of a certain operation event.

[0102] Figure 8b Detailed flow chart of step b3 is shown. Figure 8b As shown, the comparison and verification of the grayscale user's behavior data and the grayscale reporting data to obtain the operation event pass rate and the tracking pass rate (step b3) further includes:

[0103] Step d1: verifying the consistency between the grayscale user's behavior data and the event quantity and event category of the grayscale reporting data to obtain a third result;

[0104] Step d2: Verify the correctness of the sample embedding points in each event of the grayscale reporting data to obtain a fourth result;

[0105] Specifically, correctness verification can be performed based on the tracking verification rules described above.

[0106] Step d3: Calculate the tracking pass rate and the operation event pass rate based on the third result and the fourth result.

[0107] The passing rate of the buried point is calculated by the following formula:

[0108]

[0109] Among them, pr2 is the passing rate of buried points; N correct2 is the number of buried points that passed. If the third result is that the consistency verification of the number of events and event categories of the grayscale user's behavior data and the grayscale reporting data is passed, and the fourth result is that the correctness verification of the buried points in each event of the grayscale reporting data is passed, then the buried points are passed; N total2 The total number of reported burial points;

[0110] The operation event pass rate is calculated using the following formula:

[0111]

[0112] Where N is the total number of tracking points included in the operation event; PR4 is the pass rate of the operation event; p j is the pass rate of each buried point corresponding to the operation event; θ j The weight of each tracking point's pass rate as a percentage of the total event verification.

[0113] Through the above method, all tracking events can be covered, the consistency and accuracy of local operation traces and reported event tracking can be verified, and the problems of tracking in the software under test can be accurately located.

[0114] Step 22: Input the first data into the embedding quality assessment model, which is trained based on sample data. The sample data includes sample operation events, sample operation event pass rates, sample embedding points, sample embedding pass rates and sample event quality judgment results, wherein the sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standards.

[0115] Among them, the tracking quality assessment model can be completed through sample data training.

[0116] Step 23: Obtain the event quality judgment result output by the tracking quality assessment model, wherein the event quality judgment result is used to indicate whether the software to be tested corresponding to the first data meets the online standard.

[0117] The event quality judgment results output by the tracking quality assessment model are displayed in the form of enumerated categories, including direct launch, modified launch, and no launch.

[0118] The embodiment of the present invention inputs data such as operation events, operation event pass rates, tracking points and tracking point pass rates into a tracking point quality assessment model, and the tracking point quality assessment model automatically outputs event quality judgment results, indicating whether the software to be tested meets the online standards, thereby realizing automated tracking point testing, reducing the test personnel's review work on tracking point detection reports, and improving testing efficiency.

[0119] Figure 9 The flowchart of the method for training the tracking quality assessment model provided by the embodiment of the present invention is shown. Figure 9 As shown in the figure, the tracking quality assessment model can be trained through the following steps:

[0120] Step 91: Obtain sample data, the sample data including sample operation events, sample operation event pass rate, sample embedding points, sample embedding point pass rate and sample event quality judgment results, wherein the sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standards.

[0121] The sample data includes local operation data obtained by the tester through user interface UI testing of the software, as well as reported data reported by the software through the tracking software development kit SDK. The local operation data and reported data can be further used to obtain the sample operation event pass rate and the sample tracking pass rate. Based on the event pass rate and the tracking pass rate, the tester can mark the sample event quality judgment result.

[0122] In a test environment, obtaining sample data further includes:

[0123] Receive local operation data, wherein the local operation data is data obtained by a tester through performing a user interface UI test on the software;

[0124] Receive reported data, wherein the reported data is the tracking data reported by the software through the tracking software development kit SDK;

[0125] The local operation data and the reported data are compared and verified to obtain a sample event quality judgment result.

[0126] Furthermore, the comparing and verifying the local operation data and the reported data to obtain a sample event quality judgment result further includes:

[0127] Performing consistency verification on the event quantity and event category of the local operation data and the reported data to obtain a first result;

[0128] Verify whether the sample tracking points in each event of the local operation data and the reported data pass the correctness verification to obtain a second result;

[0129] Calculate the operation event pass rate, the sample embedding pass rate, and the sample operation event pass rate based on the first result and the second result;

[0130] Receive the sample event quality judgment results marked by the tester based on the event pass rate and the buried point pass rate.

[0131] The sample embedding pass rate is calculated using the following formula:

[0132]

[0133] Among them, pr1 is the sample embedding pass rate; N correct1 is the number of sample buried points that have passed. If the first result is that the consistency verification of the number of events and event categories of the local operation data and the reported data has passed, and the second result is that the correctness verification of the sample buried points in each event of the reported data has passed, then the sample buried points are passed. total1 The total number of reported sample burial points;

[0134] The pass rate of the sample operation event is calculated using the following formula:

[0135]

[0136] Where M is the total number of sample embedding points included in the sample operation event; PR3 is the pass rate of the sample operation event; p i is the passing rate of each sample embedding point corresponding to the sample operation event; θ iThe weight of each sample's passing rate as a percentage of the total sample event verification.

[0137] The processing of data during the training process can refer to the embodiment of the tracking quality assessment method of the aforementioned software, which will not be repeated here.

[0138] The training phase of the tracking quality assessment model is called the initialization phase. During this phase, tracking detection reports are output to testers and technical staff, who then score them. Scores are categorized into three categories: direct launch, modified launch, and non-launch. Testers will tolerate a certain error rate. For example, for exposure events (type: expose), the exposure element name field is not empty, but if the tracking accuracy is 98%, the tester considers it a pass and marks it as "direct launch" in the tracking detection report of the tracking verification module. For the exposure position field, the rules require it to be not empty. If the tracking accuracy is 85%, the tester considers it a failure and marks it as "modified launch." If the accuracy is only 50%, the tester marks it as "non-launch." Finally, based on the tracking pass rate of the exposure event, the tester marks the quality judgment for the exposure event.

[0139] The tracking point detection report and the corresponding events and tracking point quality judgment results can be used as training data sets to train the tracking point quality assessment model, which can be used to directly perform online level assessment on the tracking point detection report in the future, reducing the workload of testers and improving testing efficiency.

[0140] Step 92: Input the sample operation events, sample operation event pass rate, sample burial points and sample burial point pass rate in the sample data into the burial point quality assessment model to obtain a training result, wherein the training result is a training quality judgment result of the sample operation event.

[0141] Specifically, the data input into the tracking quality assessment model includes:

[0142] Total number of events - integer, for example, contains n events.

[0143] Event pass rate and quality judgment results - list data {PR1, PR2, ..., PR n}, each element is the pass rate of the corresponding event in this test.

[0144] All single event tracking pass rate - list data {pr1,pr2,…,pr m}, each element is the passing status of all the buried points of a single event in this test. Each event has m buried points, so there are ∑m i The passing rate of each buried point is used as the feature input.

[0145] During the specific implementation process, the training of the tracking point quality assessment model can adopt a deep learning classification model, and specifically models such as decision trees, random forests, and multiple classifiers can be used.

[0146] Step 93: Adjust the parameters of the embedding quality assessment model according to the training results and the sample event quality judgment results in the sample data until the error between the training results and the event quality judgment results in the sample data is less than or equal to the preset error, thereby obtaining a trained embedding quality assessment model.

[0147] As mentioned above, the model output is an enumeration category, including direct launch, modified launch, and no launch.

[0148] Furthermore, after the tester's marked tracking point quality judgment result data is recorded, the sample data is divided into a training set and a test set. The sample data includes: events (features), event pass rate (features), tracking points (features), tracking point pass rate (features), and event quality judgment results (labels).

[0149] Use the training set to train a tracking quality assessment model and apply it to the test set. When the accuracy, precision, and recall rates reach the preset range, the tracking quality assessment model is put into use.

[0150] During application, the event (feature), event pass rate (feature), buried point (feature), and buried point pass rate (feature) are given by the buried point verification module as model input, and the buried point quality assessment model automatically outputs the event quality judgment result.

[0151] Figure 10 FIG. 1 shows a schematic diagram of the structure of the software tracking quality assessment device provided by an embodiment of the present invention. Figure 10 As shown, the device 300 includes:

[0152] An acquisition module 301 is configured to acquire first data, where the first data includes operation events, operation event pass rates, tracking points, and tracking point pass rates.

[0153] Input module 302 is configured to input the first data into a tracking quality assessment model. The tracking quality assessment model is trained based on sample data. The sample data includes sample operation events, sample operation event pass rates, sample tracking points, sample tracking pass rates, and sample event quality judgment results. The sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standard.

[0154] Obtaining module 303, used to obtain the event quality judgment result output by the tracking quality assessment model, wherein the event quality judgment result is used to indicate whether the software to be tested corresponding to the first data meets the online standard.

[0155] The functions or operation steps implemented by executing the above modules are substantially the same as those in the above method embodiment and will not be repeated here.

[0156] The embodiment of the present invention inputs data such as operation events, operation event pass rates, tracking points and tracking point pass rates into a tracking point quality assessment model, and the tracking point quality assessment model automatically outputs event quality judgment results, indicating whether the software to be tested meets the online standards, thereby realizing automated tracking point testing, reducing the test personnel's review work on tracking point detection reports, and improving testing efficiency.

[0157] Figure 11 A structural schematic diagram of the computing device provided by an embodiment of the present invention is shown, wherein the computing device can be a computer or server or other device that provides software tracking quality assessment for each client. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0158] like Figure 11 As shown, the computing device may include: a processor 402 , a communications interface 404 , a memory 406 , and a communication bus 408 .

[0159] Processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408. Communication interface 404 is used to communicate with other devices, such as clients or other server network elements. Processor 402 is used to execute program 410, which may specifically perform the relevant steps of the aforementioned software tracking quality assessment method or the method for training a tracking quality assessment model.

[0160] Specifically, the program 410 may include program code including computer-executable instructions.

[0161] Processor 402 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The one or more processors included in a computing device may be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0162] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0163] The embodiment of the present invention inputs data such as operation events, operation event pass rates, tracking points and tracking point pass rates into a tracking point quality assessment model, and the tracking point quality assessment model automatically outputs event quality judgment results, indicating whether the software to be tested meets the online standards, thereby realizing automated tracking point testing, reducing the test personnel's review work on tracking point detection reports, and improving testing efficiency.

[0164] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is run on a computing device, the computing device executes the software tracking quality assessment method or the method for training a tracking quality assessment model in any of the above-mentioned method embodiments.

[0165] An embodiment of the present invention provides a software tracking quality assessment device, which is used to execute the above-mentioned software tracking quality assessment method or the method of training a tracking quality assessment model.

[0166] An embodiment of the present invention provides a computer program that can be called by a processor to enable a computing device to execute the software tracking quality assessment method or the method for training a tracking quality assessment model in any of the above method embodiments.

[0167] An embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are run on a computer, the computer executes the software tracking quality assessment method or the method for training a tracking quality assessment model in any of the above method embodiments.

[0168] The algorithm or demonstration provided herein are not inherently relevant to any particular computer, virtual system or other equipment. Various general-purpose systems may also be used together with the teachings based on this. According to the above description, it is apparent that the structure required for constructing this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the above description of specific languages ​​is for the purpose of disclosing the best mode of the present invention.

[0169] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0170] Similarly, it should be understood that in order to streamline the present invention and facilitate understanding of one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim.

[0171] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed so far can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0172] It should be noted that the above embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by a person skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between brackets should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising several different elements and by means of appropriately programmed computers. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments should not be understood as limiting the order of execution unless otherwise specified.

Claims

1. A method for evaluating the quality of software tracking points, characterized in that: The method comprises: Obtaining first data, where the first data includes an operation event, an operation event pass rate, a tracking point, and a tracking point pass rate. The operation event includes at least one field, and the tracking point pass rate is for a single field. Input the first data into a tracking point quality assessment model, where the tracking point quality assessment model is trained based on sample data. The sample data includes sample operation events, sample operation event pass rates, sample tracking points, sample tracking point pass rates, and sample event quality judgment results. The sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standards. The sample operation event includes at least one field, and the sample tracking point pass rate is for a single field. The tracking point verification rules configured for the event are used to verify whether the tracking point in each operation event passes. Obtain an event quality judgment result output by the tracking quality assessment model, wherein the event quality judgment result is used to indicate whether the software under test corresponding to the first data meets the online standard.

2. The method according to claim 1, characterized in that The tracking quality assessment model is trained in the following way: obtaining the sample data; Input the sample operation events, sample operation event pass rates, sample buried points, and sample buried point pass rates in the sample data into a buried point quality assessment model to obtain a training result, wherein the training result is a training quality judgment result of the sample operation events; According to the training results and the sample event quality judgment results in the sample data, the parameters of the embedding quality assessment model are adjusted until the error between the training results and the event quality judgment results in the sample data is less than or equal to the preset error, thereby obtaining a trained embedding quality assessment model.

3. The method according to claim 2, characterized in that The sample data includes local operation data obtained by the tester through the user interface UI test of the software, and the reported data reported by the software through the tracking software development kit SDK; The first data includes the behavior data of grayscale users in the grayscale verification environment and the grayscale reporting data reported by the software through the tracking software development kit SDK in the grayscale verification environment.

4. The method according to claim 3, characterized in that The method further comprises: Performing consistency verification on the event quantity and event category of the local operation data and the reported data to obtain a first result; Verifying the correctness of the sample embedding points in each event of the reported data to obtain a second result; Calculate the sample embedding pass rate and the sample operation event pass rate based on the first result and the second result; or, The method further comprises: Performing consistency verification on the grayscale user's behavior data and the event quantity and event category of the grayscale reporting data to obtain a third result; Verify the correctness of the sample embedding points in each event of the grayscale reporting data to obtain a fourth result; The point embedding pass rate and the operation event pass rate are calculated based on the third result and the fourth result.

5. The method according to claim 4, characterized in that The sample embedding pass rate is calculated using the following formula: in, The passing rate of sample embedding; is the number of sample burial points that have passed. If the first result is that the consistency verification of the number of events and event categories of the local operation data and the reported data has passed, and the second result is that the correctness verification of the sample burial points in each event of the reported data has passed, then the sample burial points are passed. The total number of reported sample burial points; The embedding pass rate is calculated by the following formula: in, The passing rate of buried points; is the number of tracking points that passed. If the third result is that the consistency verification of the number of events and event categories of the grayscale user's behavior data and the grayscale reporting data is passed, and the fourth result is that the correctness verification of the tracking points in each event of the grayscale reporting data is passed, then the tracking point is a passed tracking point. The total number of reported burial points; The pass rate of the sample operation event is calculated using the following formula: Where M is the total number of sample embedding points included in the sample operation event; is the passing rate of sample operation events; The pass rate of each sample tracking point corresponding to the sample operation event; The weight of each sample's tracking pass rate as a percentage of the total sample event verification; The operation event pass rate is calculated using the following formula: Where N is the total number of tracking points included in the operation event; is the operation event pass rate; The pass rate of each tracking point corresponding to the operation event; The weight of each tracking point's pass rate as a percentage of the total event verification.

6. A method for training a tracking quality assessment model, characterized in that: The method comprises: Obtain sample data, including sample operation events, sample operation event pass rates, sample tracking points, sample tracking point pass rates, and sample event quality judgment results. The sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standards. The sample operation event contains at least one field. The sample tracking point pass rate is for a single field. Use the tracking point verification rules configured for the event to verify whether the tracking point in each operation event passes. Input the sample operation events, sample operation event pass rates, sample buried points, and sample buried point pass rates in the sample data into a buried point quality assessment model to obtain a training result, wherein the training result is a training quality judgment result of the sample operation events; According to the training results and the sample event quality judgment results in the sample data, the parameters of the embedding quality assessment model are adjusted until the error between the training results and the event quality judgment results in the sample data is less than or equal to the preset error, thereby obtaining a trained embedding quality assessment model.

7. A software tracking quality assessment device, characterized in that: The device comprises: An acquisition module is configured to acquire first data, the first data including an operation event, an operation event pass rate, a buried point, and a buried point pass rate, wherein an operation event includes at least one field, and the buried point pass rate is for a single field; An input module is configured to input the first data into a tracking point quality assessment model, wherein the tracking point quality assessment model is trained based on sample data, wherein the sample data includes sample operation events, sample operation event pass rates, sample tracking points, sample tracking point pass rates, and sample event quality judgment results, wherein the sample event quality judgment results are used to indicate whether the software corresponding to the sample data meets the online standard, wherein the sample operation event includes at least one field, the sample tracking point pass rate is for a single field, and the tracking point verification rules configured for the event are used to verify whether the tracking point in each operation event passes; An obtaining module is used to obtain the event quality judgment result output by the tracking quality assessment model, wherein the event quality judgment result is used to indicate whether the software under test corresponding to the first data meets the online standard.

8. A computing device, characterized in that include: A processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The storage medium stores at least one executable instruction, and when the executable instruction is executed on the computing device, the computing device executes the operation of the method according to any one of claims 1 to 6.

Images