Gray scale test method, device and equipment for popularization resource integration package

Abstract

The invention provides a gray scale testing method, device and equipment for a promotion resource integration package, and belongs to the field of software testing. According to the invention, before the popularization resource integration package is formally published, the double-device comparison test is carried out on the popularization resource integration package. Wherein a to-be-tested version of the promotion resource integration package is installed on one of the two devices, and a formal version of the promotion resource integration package is installed on the other device, and the test mode realizes that in a test environment, a server directly controls data input of the two devices; and the difference of the promotion resource integration packages of different versions is verified by running the promotion resource integration packages of different versions in parallel through double devices. And the server side can quickly locate the function and stability problems of the version to be tested by comparing the execution results of the two devices, and directly repairs the problems in the test environment without waiting for the auditing of an application store. The use experience of the to-be-tested version on the user side after the to-be-tested version is comprehensively online is ensured, and the use risk of the user side is reduced.

Description

Technical Field

[0001] This disclosure relates to the field of software testing, and in particular to a gray-scale testing method, apparatus, and equipment for promoting resource integration packages. Background Technology

[0002] Promotional resource integration packages (PUAs) are a set of code tools embedded in an application to help developers integrate promotional functionality into their applications. Typically, PUAs are embedded into a developer's application as a third-party SDK (Software Development Kit).

[0003] In related technologies, because iOS does not have plug-in capabilities, iOS promotional resource integration packages cannot bypass the app store's review and be dynamically updated. This means that traditional gray-scale testing solutions are not feasible for iOS promotional resource integration packages, because traditional gray-scale testing solutions rely on plug-in technology to dynamically update SDK functions.

[0004] Because traditional gray-scale testing methods are not feasible, it is impossible to recall and fix issues before the official release. Therefore, once the new version's promotional resource integration package is fully launched, any functional or stability issues will affect a massive number of users. At this point, it is not possible to quickly fix the problem by dynamically loading plugins, as it requires waiting for app store approval. Therefore, how to conduct gray-scale testing on the iOS promotional resource integration package to verify the functionality and stability of the new SDK version in advance has become a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] This disclosure provides a method, apparatus, and device for grayscale testing of promotional resource integration packages. The technical solution of this disclosure is as follows:

[0006] According to a first aspect of the present disclosure, a gray-scale testing method for promoting a resource integration package is provided, comprising:

[0007] The system retrieves promotional resource configuration data from a data source and sends the same promotional resource configuration data to both the first terminal and the second terminal. The first terminal has a target application with an embedded official version of the promotional resource integration package, while the second terminal has a target application with an embedded test version of the promotional resource integration package. The promotional resource configuration data is used to instruct the first terminal and the second terminal to display promotional resources in the same display position.

[0008] The first terminal and the second terminal are controlled to execute the same test cases; wherein, when the test cases are executed, they are used to simulate user operations according to a preset operation path, and the preset operation path is related to the type of promotional resources displayed;

[0009] Obtain the first execution result reported by the first terminal and the second execution result reported by the second terminal;

[0010] Compare the first execution result and the second execution result on at least one metric, and output the test result of the version to be tested based on the difference.

[0011] In some embodiments, the method further includes:

[0012] Acquire the first type of promotion resource allocation data collected in the online production environment;

[0013] Obtain the second type of promotional resource configuration data generated in the test environment; wherein, the second type of promotional resource configuration data is simulated data generated by recording user behavior in the test environment;

[0014] The data source is obtained by merging the first type of promotional resource configuration data with the second type of promotional resource configuration data.

[0015] In other embodiments, the step of acquiring the first type of promotional resource configuration data collected in the online production environment includes: acquiring the original promotional resource configuration data collected in the online production environment;

[0016] Based on the first whitelist, the original promotional resource configuration data is filtered to obtain the first type of promotional resource configuration data.

[0017] In other embodiments, obtaining the second type of promotional resource configuration data generated in the test environment includes:

[0018] Record user behavior in real time in the test environment to generate simulated promotion resource configuration data;

[0019] Based on the testing requirements, the simulated promotion resource configuration data is filtered for the first time.

[0020] Based on the second whitelist, the simulated promotion resource configuration data that passed the first filter is filtered a second time to obtain the second type of promotion resource configuration data.

[0021] In other embodiments, the preset operation path includes multiple operation steps; the test case, when executed, is used to simulate user operations according to the preset operation path, including:

[0022] The test cases are used to perform random interactive operations on valid page elements when executing the operation steps; wherein, the valid page elements are used to guide the object to perform interactive operations.

[0023] In other embodiments, the preset operation path includes multiple operation steps; the test case, when executed, is used to simulate user operations according to the preset operation path, including:

[0024] The test case, when executing the operation steps, is used to perform interactive operations based on coordinate data collected in the online production environment; wherein, the coordinate data includes at least one of the following:

[0025] Coordinate data matching the device parameters of the first terminal and the second terminal;

[0026] Coordinate data collected within a preset time period.

[0027] In other embodiments, obtaining the first execution result reported by the first terminal and the second execution result reported by the second terminal includes:

[0028] The system receives the first execution result reported by the first terminal through a data collection point embedded in the official version; wherein the first execution result carries a first type identifier, which is used to indicate that the data originates from the official version.

[0029] The second execution result reported by the second terminal through the data collection point embedded in the version under test is accepted; wherein the second execution result carries a second type identifier, which is used to indicate that the data comes from the version under test.

[0030] In other embodiments, the method further includes:

[0031] A new field is added to the data collection point; wherein, when the new field is executed on the first terminal or the second terminal, it is responsible for collecting and reporting data for statistical analysis of the new indicators.

[0032] In other embodiments, the method further includes:

[0033] The promotion resource configuration data included in the data source is matched with the rules included in the rule set to obtain the rule matching result;

[0034] Perform a rule count check, which is used to verify whether the rule matching result conforms to the coverage of the rule declaration;

[0035] After the rules that pass the verification are converted into test cases, they are stored in the test case library to form a test case set.

[0036] In other embodiments, the method further includes:

[0037] Based on the test objectives and test scenarios, rules are automatically generated for different test data types to obtain standardized rules;

[0038] Obtain manually entered rules to get non-standardized rules;

[0039] The standardized rules and the non-standardized rules are combined to obtain the rule set.

[0040] In other embodiments, the method further includes:

[0041] For any rule in the rule set, the coverage rate of the rule is calculated; wherein, the coverage rate of the rule refers to the ratio of the amount of data in the data source that actually triggers the rule to the total amount of data that may trigger the rule;

[0042] If the coverage of the rule is less than the coverage threshold, a target operation is performed, which includes:

[0043] Adjust the rules; or,

[0044] Expand the data range covered by the rules.

[0045] In other embodiments, the method further includes:

[0046] If the coverage of the rule is less than the coverage threshold, and the rule is a critical rule, an alarm operation is performed;

[0047] The key rules are used to monitor the preset functions or preset indicators of the promotion resource integration package.

[0048] According to a second aspect of the present disclosure, a grayscale testing apparatus for promoting a resource integration package is provided, comprising:

[0049] The distribution module is configured to obtain promotional resource configuration data from a data source and distribute the same promotional resource configuration data to a first terminal and a second terminal; wherein, the target application installed on the first terminal has an official version of the promotional resource integration package embedded in it, and the target application installed on the second terminal has a test version of the promotional resource integration package embedded in it; the promotional resource configuration data is used to instruct the first terminal and the second terminal to display promotional resources in the same display position;

[0050] The control module is configured to control the first terminal and the second terminal to execute the same test cases; wherein, when the test cases are executed, they are used to simulate user operations according to a preset operation path, and the preset operation path is related to the type of promotional resources displayed;

[0051] The acquisition module is configured to acquire the first execution result reported by the first terminal and the second execution result reported by the second terminal;

[0052] The analysis module is configured to compare the first execution result and the second execution result on at least one metric;

[0053] The output module is configured to output the test results of the version under test based on the differences.

[0054] In some embodiments, the apparatus further includes:

[0055] The construction module is configured to acquire first-type promotional resource configuration data collected in an online production environment; acquire second-type promotional resource configuration data generated in a test environment; wherein the second-type promotional resource configuration data is simulated data generated by recording user behavior in the test environment; and merge the first-type promotional resource configuration data and the second-type promotional resource configuration data to obtain the data source.

[0056] In other embodiments, the building module is configured as follows:

[0057] Obtain raw promotional resource configuration data collected in an online production environment;

[0058] Based on the first whitelist, the original promotional resource configuration data is filtered to obtain the first type of promotional resource configuration data.

[0059] In other embodiments, the building module is configured as follows:

[0060] Record user behavior in real time in the test environment to generate simulated promotion resource configuration data;

[0061] Based on the testing requirements, the simulated promotion resource configuration data is filtered for the first time.

[0062] Based on the second whitelist, the simulated promotion resource configuration data that passed the first filter is filtered a second time to obtain the second type of promotion resource configuration data.

[0063] In other embodiments, the preset operation path includes multiple operation steps; the test case, when executed, is used to simulate user operations according to the preset operation path, including:

[0064] The test cases are used to perform random interactive operations on valid page elements when executing the operation steps; wherein, the valid page elements are used to guide the object to perform interactive operations.

[0065] In other embodiments, the preset operation path includes multiple operation steps; the test case, when executed, is used to simulate user operations according to the preset operation path, including:

[0066] The test case, when executing the operation steps, is used to perform interactive operations based on coordinate data collected in the online production environment; wherein, the coordinate data includes at least one of the following:

[0067] Coordinate data matching the device parameters of the first terminal and the second terminal;

[0068] Coordinate data collected within a preset time period.

[0069] In other embodiments, the acquisition module is configured to:

[0070] The system receives the first execution result reported by the first terminal through a data collection point embedded in the official version; wherein the first execution result carries a first type identifier, which is used to indicate that the data originates from the official version.

[0071] The second execution result reported by the second terminal through the data collection point embedded in the version under test is accepted; wherein the second execution result carries a second type identifier, which is used to indicate that the data comes from the version under test.

[0072] In other embodiments, the device further includes:

[0073] The first processing module is configured to add a field to the data collection point; wherein, when the added field is executed on the first terminal or the second terminal, it is responsible for collecting and reporting data for statistical analysis of the new indicators.

[0074] In other embodiments, the device further includes:

[0075] The rule matching module is configured to match the promotion resource configuration data included in the data source with the rules included in the rule set to obtain the rule matching result;

[0076] The verification module is configured to perform rule count verification, which is used to verify whether the matching result of the verification rule conforms to the coverage of the rule declaration;

[0077] The storage module is configured to convert the validated rule matching results into test cases and store them in the test case library to form a test case set.

[0078] In other embodiments, the device further includes:

[0079] The generation module is configured to automatically generate rules for different test data types based on the test objectives and test scenarios to obtain standardized rules; obtain manually entered rules to obtain non-standardized rules; and merge the standardized rules and the non-standardized rules to obtain the rule set.

[0080] In other embodiments, the device further includes:

[0081] The statistics module is configured to calculate the coverage rate of any rule in the rule set; wherein the rule coverage rate refers to the ratio of the amount of data in the data source that actually triggers the rule to the total amount of data that may trigger the rule;

[0082] The second processing module is configured to perform a target operation if the coverage of the rule is less than a coverage threshold. The target operation includes: adjusting the rule; or expanding the data range covered by the rule.

[0083] In other embodiments, the device further includes:

[0084] The third processing module is configured to perform an alarm operation if the coverage rate of the rule is less than the coverage rate threshold and the rule is a critical rule; wherein the critical rule is used to monitor the preset functions or preset indicators of the promotion resource integration package.

[0085] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0086] One or more processors;

[0087] Memory used to store the executable program code of the processor;

[0088] The processor is configured to execute the program code to implement the grayscale testing method described above.

[0089] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein program code in the computer-readable storage medium is executed by a processor of an electronic device, enabling the electronic device to perform the grayscale testing method described above.

[0090] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor of an electronic device, implements the grayscale testing method described above.

[0091] The gray-scale testing scheme provided in this disclosure enables dual-device comparative testing of the promotional resource integration package for iOS before its official release. This scheme bypasses iOS's plugin limitations, achieving testing results similar to traditional gray-scale testing schemes. Specifically, one device in the dual-device setup has the test version of the promotional resource integration package (also known as the gray-scale version) installed, while the other device has the official version (also known as the gray-scale comparison version) installed. This testing mode allows the server to directly control the data input of both devices in a test environment isolated from the online production environment. By running different versions of the SDK in parallel on both devices, the differences in various metrics between the different SDK versions are verified. By comparing the execution results of the two devices, the server can quickly locate functional and stability issues in the gray-scale version and fix them directly in the test environment without waiting for app store approval. In summary, this scheme enables the early detection of potential functional and stability issues in the new SDK version before its official release, thereby ensuring a better user experience after the new SDK version is fully launched and reducing user-side risks.

[0092] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0093] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0094] Figure 1 This is a schematic diagram illustrating the implementation environment of a gray-scale testing method for a promotional resource integration package according to an exemplary embodiment.

[0095] Figure 2 This is a schematic diagram illustrating the execution flow of a dual-device comparative test for promoting a resource integration package, according to an exemplary embodiment.

[0096] Figure 3 This is a flowchart illustrating a grayscale testing method for a promotional resource integration package according to an exemplary embodiment.

[0097] Figure 4 This is a schematic diagram illustrating a process for constructing a data source, rule set, and use case library according to an exemplary embodiment.

[0098] Figure 5 This is a schematic diagram illustrating the entire process of promotional resource playback and user conversion according to an exemplary embodiment.

[0099] Figure 6This is a block diagram illustrating a grayscale testing apparatus for a promotional resource integration package according to an exemplary embodiment.

[0100] Figure 7 This is a block diagram illustrating a server according to an exemplary embodiment. Detailed Implementation

[0101] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0102] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0103] The information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by the parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0104] Figure 1 This is a schematic diagram illustrating the implementation environment of a gray-scale testing method for a promotional resource integration package according to an exemplary embodiment. See also Figure 1 The implementation environment includes: a first terminal 101, a second terminal 102, a server 103, and a data visualization platform 104.

[0105] The first terminal 101 has a target application with a promotional resource integration package embedded in its official version, while the second terminal 102 has a target application with a promotional resource integration package embedded in its test version.

[0106] In this embodiment of the disclosure, the target application is any developer APP that can be embedded with a promotional resource integration package, such as a short video application, and this disclosure does not limit it.

[0107] It should be noted that for promotional resource integration packages, the released and online version is referred to as the official version or the gray-scale comparison version. While the official version is running, developers may develop a new version of the promotional resource integration package for various reasons (such as fixing issues or adding new features). Before the official release of the new version, testers can verify its functionality and stability through dual-device comparison testing. In this document, the new version is referred to as the test version or the gray-scale version, that is, the version participating in gray-scale testing before the official release. Correspondingly, the first terminal 101 is also referred to as the online device, and the second terminal 102 is also referred to as the gray-scale device.

[0108] In some embodiments, the terminal 101 is a device such as a smartphone or tablet. Figure 1 The example only uses the first terminal 101 and the second terminal 102 as smartphones.

[0109] In other embodiments, server 103 is at least one of a PC, a physical server, multiple physical servers, a cloud computing platform, and a virtualization center. Data visualization platform 104 can be part of server 103 or deployed on other devices, connected to server 103 via wired or wireless means; this disclosure does not limit this.

[0110] In addition, the server 103 connects to the first terminal 101 and the second terminal 102 via a wireless network or a wired network. By sending data or operation instructions to the first terminal 101 and the second terminal 102, the server enables the first terminal 101 and the second terminal 102 to run different versions of the promotional resource integration package in parallel, thereby achieving dual-device comparison testing.

[0111] The grayscale testing scheme provided in this disclosure will be described in detail below through the following embodiments.

[0112] Figure 2 This is a schematic diagram illustrating the execution flow of a dual-device comparative test for promoting a resource integration package, according to an exemplary embodiment. See also... Figure 2 This process involves four parts: distributing promotional resource configuration data, embedding different versions of promotional resource integration packages on both devices, simulating user operations, and inspection and construction. These four parts will be described in detail below.

[0113] Distribute promotional resource allocation data.

[0114] In this embodiment, the promotional resource configuration data sent by the server to both devices comes from the online production environment; that is, real data is captured from the real environment, processed, and then used to generate a reusable test dataset. This dataset can reproduce the complexity of real-world scenarios and support the diversity of gray-scale testing. See also Figure 2 The promotion of resource allocation data includes the following three dimensions: data coverage, data validity, and data storage.

[0115] The data coverage includes marketing objectives, types of promotional resources, and conversion goals. Marketing objectives refer to the core purpose of the promotion, such as brand exposure, user downloads, or product conversions. Types of promotional resources include, but are not limited to, banner ads, interstitial ads, rewarded videos, and in-feed ads. Conversion goals are also known as user behavior tracking goals, such as clicks, registrations, or purchases.

[0116] Regarding data validity, measures are taken to ensure the real-time nature and accuracy of data by filtering invalid or expired device data. The expiration rule automatically cleans up expired data (such as click data from one month ago) to avoid interfering with real-time analysis. Device filtering filters device data based on device attributes (such as iOS / Android, device model, or operating system version).

[0117] For data storage, this defines the storage method and update frequency. Hourly caching refers to caching data at an hourly granularity, meaning data is updated once per hour to balance real-time performance and storage costs.

[0118] Embed different versions of the promotional resource integration package on both devices.

[0119] In this embodiment, both devices operate in a test environment, completely isolated from the online production environment, to avoid impacting real users. Furthermore, to ensure consistency in the test environment, both devices receive the same mock data to eliminate external interference. Additionally, the grayscale version and the grayscale comparison version of the SDK are embedded in the same host app and bound to the same display position, again to ensure the uniqueness of test environment variables, with only version differences, thus eliminating interference from other factors.

[0120] It should be noted that the SDK mentioned in this article refers to the promotional resource integration package.

[0121] Technical data tracking and reporting refers to collecting runtime data from the grayscale version and the comparison version of the SDK to provide a basis for performance comparison. Additionally, the data reported through technical data tracking and reporting includes an identifier field to distinguish the data source; for example, `version=1` indicates the data comes from the grayscale version, and `version=2` indicates the data comes from the comparison version. Furthermore, this solution supports adding technical data tracking and reporting in a customized manner to achieve automatic configuration of monitoring metrics.

[0122] Simulate user actions.

[0123] To trigger more effective actions and reduce ineffective actions (such as clicking on blank areas or non-critical controls) within a limited time (e.g., a single grayscale period), this solution employs a customized UI automation approach. The core of this customized UI automation solution is to implement a better click strategy, which supports two methods: random operation and online recording and playback. Random operation involves rule-guided random clicks, improving accuracy through page element positioning. Online recording and playback simulates user actions based on real click coordinates to enhance click effectiveness. Click effectiveness refers to the proportion of users who click on promotional resources and trigger the expected behavior (e.g., download or registration).

[0124] In other words, this solution optimizes the click strategy to simulate user actions that change from completely random to rule-guided randomness or reproduction of real-world behavior.

[0125] To address random clicks guided by rules, pre-standardize naming conventions. For example, define a unique set of page IDs for each page (such as a playback page, end page, or landing page). In addition, standardize page elements, such as naming controls on each page according to a unified rule to ensure that targets can be exhaustively identified. Then, effective conversion can be achieved by specifying control IDs (such as close buttons or skip buttons), recognizing image features (such as banner images or video covers), recognizing text content (such as copy), and specifying area coordinates (such as bottom banners), thus avoiding invalid clicks.

[0126] In summary, specifying a region refers to limiting the clickable area by dividing the page into sections. Specifying a control, image, or text refers to accurately locating the effective click target based on a uniformly named control ID, image characteristics, or text content (such as "Download Now").

[0127] For online recording and playback, real click coordinates are obtained from the online user behavior data warehouse. These coordinates are then used to reproduce the click paths of real users in the test environment, simulating high-value behaviors. Furthermore, since iOS device models and screen sizes are enumerable, for a specific device, the corresponding real click coordinates can be retrieved from the online user data warehouse table, ensuring that the playback coordinates match the actual device. In other words, in the test environment, the functionality and stability of the version under test are verified by replaying previously recorded real user click data.

[0128] In summary, filtering click behavior for specific device models refers to extracting the actual click coordinates of specific devices from the online user behavior data warehouse to ensure that the replayed actions are consistent with those of real users. Hourly data replay refers to replaying user click paths at a time granularity (such as the click hotspots of a landing page during a certain period).

[0129] Inspection and construction.

[0130] See Figure 2 The inspection dimensions include overall metrics (global perspective), business scenario inspection (refined analysis), anomaly monitoring (problem discovery), and functional point monitoring (root cause identification). Overall metrics allow us to understand the health of core businesses; business scenario inspections and anomaly monitoring quickly identify issues affecting user experience; and functional point monitoring ensures the stability of core processes (such as exposure and clicks). Below is an explanation of some terms and concepts related to the inspection system.

[0131] Launch success rate refers to the percentage of successful initialization of the promotion resource integration package; CTR (Click Through Rate) refers to the ratio of the number of times a user actually clicks on a promotion resource to the number of times the promotion resource is displayed; CVR (Conversion Rate) refers to the percentage of users who complete a conversion (such as downloading, registering, or purchasing) after clicking on a promotion resource; Request success rate refers to the percentage of times the SDK successfully initiates a request to the promotion resource platform; Exposure refers to the number of times an ad is displayed; Conversion funnel refers to the conversion rate of the entire link from exposure to conversion; Crash rate refers to the frequency of SDK crashes; ANR (Application Not Response) refers to the main thread blocking problem caused by promotion resources; Style delivery success rate refers to the success rate of loading materials (such as banner images and videos); Callback interface success rate is used to reflect the stability of interfaces such as redirection and data return.

[0132] In summary, the gray-scale testing scheme provided in this disclosure enables dual-device comparative testing of the iOS SDK before its official release. This scheme bypasses iOS's plugin limitations, achieving testing results similar to traditional gray-scale testing schemes. Specifically, one device in the dual-device setup has the gray-scale version of the SDK installed, while the other device has the official version. This testing mode allows the server to directly control the data input of both devices in a test environment isolated from the online production environment. By running different versions of the SDK in parallel on both devices, the differences in various metrics between the different SDK versions can be verified. By comparing the execution results of the two devices, the server can quickly locate functional and stability issues in the gray-scale version and fix them directly in the test environment without waiting for app store approval. This scheme enables the early detection of potential functional and stability issues in the new SDK version before its official release, thereby ensuring a better user experience after the new SDK version is fully launched and reducing user-side risks.

[0133] Figure 3 This is a flowchart illustrating a gray-scale testing method for a promotional resource integration package according to an exemplary embodiment. For example... Figure 3As shown, this grayscale testing method is applied to electronic devices, such as... Figure 1 The server 103 is shown. This grayscale testing method includes the following steps:

[0134] In 301, the server obtains promotional resource configuration data from the data source and sends the same promotional resource configuration data to the first terminal and the second terminal. The target application installed on the first terminal contains the official version of the promotional resource integration package, while the target application installed on the second terminal contains the test version of the promotional resource integration package. The promotional resource configuration data is used to instruct the first terminal and the second terminal to display promotional resources in the same display position.

[0135] The data distributed by the server is crucial for testing the integrated promotional resource package. Without distribution restrictions, promotional resources distributed to individual devices are random, potentially leading to issues with limited data types. This results in limited functionality triggered on offline devices (physical devices like smartphones or tablets) (uncontrollable real-world user behavior) and incomplete branch coverage (fixed user operation paths fail to cover abnormal branches; lack of testing for edge scenarios), making it difficult to identify problems during gray-scale testing. To address this, this solution records diverse, real-world online data and ensures the timeliness of promotional resource configuration data to prevent resource unavailability. Furthermore, this solution performs dual-device comparison testing, ensuring consistent promotional resource configuration data distributed to both devices simultaneously. The distributed data covers different marketing objectives, data types, and conversion goals, and is distributed to both devices according to the online deployment ratio, significantly improving data simulation accuracy.

[0136] In summary, gray-scale testing requires verifying the performance of the new SDK in real user scenarios. Due to the short gray-scale testing cycle, in order to accumulate enough data within a limited time to cover multiple scenarios, this disclosure embodiment improves the data simulation of promotion resource configuration data through the following methods.

[0137] Rule set generation.

[0138] See Figure 4 When generating rule sets, the server employs both component-based and non-component-based methods. For the component-based method, applicable data types include basic data (such as the ID and location of promotional resources), enumerations (such as the type of promotional resources), and rendering templates. It can parse style libraries and component libraries, and then automatically generate standardized rules according to preset logic. For the non-component-based method, applicable data types include switches (such as enabling / disabling promotional resources) and marketing objectives. Rule logic can be manually entered, such as forcing a redirect to an H5 landing page during a specific campaign.

[0139] As an example, based on the above description, a rule set can be generated as follows:

[0140] On the one hand, based on the test objectives (also known as test requirements, which can be determined through the current test case set) and test scenarios (such as exposure control, click verification, or conversion rate analysis), rules are automatically generated for different test data types (such as basic data, switches, enumeration numbers, rendering templates, or marketing objectives) to obtain standardized rules; on the other hand, manually entered rules are obtained to obtain non-standardized rules; then, standardized rules and non-standardized rules are merged to obtain a rule set.

[0141] As another example, standardized rules generated through componentization and special rules generated through non-componentization can be merged into a complete rule set through tag matching to ensure coverage of all test scenarios. Furthermore, the final rule set obtained after merging is entered into the rule library, supporting dynamic invocation and version iteration.

[0142] Data source construction.

[0143] In some embodiments, see Figure 4 This disclosed embodiment constructs a data source by working collaboratively in two environments (online production environment and PRT environment):

[0144] 3011. Obtain the first type of promotion resource allocation data collected in the online production environment.

[0145] For this step, the server obtains the original promotional resource configuration data collected in the online production environment; then, based on the first whitelist, it filters the original promotional resource configuration data to obtain the first type of promotional resource configuration data.

[0146] This is designed for online production environment processes, used to collect production environment data in real time and filter invalid data using a whitelist. The following section combines... Figure 5 It will be described in detail.

[0147] See Figure 5 HIVE serves as a data warehouse, storing raw promotional resource configuration data, while Kafka Topics (message queues) act as real-time data pipelines, receiving raw data pushed from HIVE via API. The server, acting as a consumer, obtains the raw data stream by subscribing to a specified Topic. Subsequently, the server filters the data based on a first whitelist, retaining only data that meets the criteria, thus obtaining the first type of promotional resource configuration data.

[0148] For example, the first whitelist is also referred to as the online whitelist, which includes posid (display ID) and aupid (ad platform ID), and this disclosure does not limit this.

[0149] 3012. Obtain the second type of promotion resource configuration data generated in the test environment; wherein, the second type of promotion resource configuration data is simulated data generated by recording user behavior in the test environment.

[0150] For the PRT environment process, it is used to simulate or supplement the data obtained in step 3011 to adapt to test requirements.

[0151] See Figure 4 Real-time recording refers to recording user behavior in a PRT environment to generate simulated data. Selective storage refers to filtering the recorded data according to testing requirements, such as retaining only crash scenarios or specific types of data.

[0152] Based on the above description, obtaining the second type of promotion resource configuration data generated in the test environment includes, but is not limited to, the following methods: recording user behavior in real time in the test environment to generate simulated promotion resource configuration data; performing a first filter on the simulated promotion resource configuration data according to test requirements; and then, performing a second filter on the simulated promotion resource configuration data that passed the first filter according to the second whitelist to obtain the second type of promotion resource configuration data.

[0153] For example, the second whitelist includes posid (display ID), uid (user ID), and appid (application ID), but this disclosure does not limit this.

[0154] 3013. Merge the first type of promotion resource allocation data with the second type of promotion resource allocation data to obtain the data source.

[0155] When merging data, you can perform actions such as data deduplication, data validation (e.g., checking field integrity), or tagging data according to its source to achieve tagged storage. This disclosure does not limit these actions.

[0156] Use case set construction.

[0157] The following is combined Figure 4 Let me introduce the process of building a test case set.

[0158] As mentioned earlier, the rule base stores a predefined set of rules, providing the logical basis for test case generation. Rule matching is used to match the promotion resource configuration data included in the data source with the rules included in the rule set, thereby obtaining the rule matching result. As an example, the matching modes include one-to-one matching and one-to-many matching. One-to-one matching refers to precisely matching a single rule, that is, a single rule corresponds to a single piece of data. One-to-many matching is when a single rule covers multiple data scenarios, that is, a single rule can match multiple pieces of data.

[0159] After rule matching is completed, rule count verification begins. This verification checks whether the rule matching results conform to the coverage of the rule declarations in the rule set. Finally, the rule matching results that pass the verification are converted into test cases and stored in the test case library, forming a test case set.

[0160] Data update mechanism.

[0161] See Figure 4 The data update mechanism combines automated updates with manual intervention to ensure data timeliness while supporting flexible data adjustments. Automated updates refer to periodically updating matching tags and dynamic templates through scheduled tasks. Manual intervention involves providing a switch that allows users to stop scheduled tasks at any time and manually update matching tags or dynamic templates to handle unexpected scenarios (such as the urgent launch of an event page) and avoid waiting for the scheduled task cycle to complete.

[0162] The scheduled tasks start at preset intervals, performing updates to matching tags and dynamic templates respectively. Updating matching tags refers to updating tags used for data matching (such as targeting tags and user profile tags). Updating dynamic templates refers to updating dynamic content such as material templates and landing page templates (such as event page styles).

[0163] Additionally, responding to a matching tag update event may also involve adjusting the rules included in the rule set. For example, if the targeting conditions need to change with user behavior data (such as adding a "high-value user" tag), a scheduled task will fetch the latest tag configuration, update the tag matching logic in the rule base, and apply the new rules. Furthermore, responding to a dynamic template update event may also involve re-executing the rule count check.

[0164] Coverage statistics and optimization.

[0165] Coverage statistics quantify the degree of coverage between the rule set and the data source, identifying rules not covered by data or data not covered by rules. Intelligent whitelisting automatically supplements missing data or rules to improve test coverage. High-priority rule alerts monitor the coverage of key rules in real time and trigger alerts. Performance optimization optimizes data processing workflows, improves data processing efficiency, and reduces resource consumption. For example, frequently accessed rules can be cached in memory to reduce database query pressure; time-consuming operations can be moved to background tasks to avoid blocking the main process.

[0166] In this embodiment, for any rule in the rule set, the server calculates the coverage rate of that rule. The coverage rate refers to the ratio of the amount of data in the data source that actually triggers the rule to the total amount of data that could potentially trigger it. High coverage indicates that the rule can accurately capture anomalies. Low coverage may be due to overly strict rule threshold settings or data noise interference.

[0167] If the coverage of this rule is less than the coverage threshold, execute the target operation. For example, the target operation includes: adjusting the rule; or, expanding the data range covered by the rule. Adjusting the rule refers to optimizing the rule parameters, i.e., adjusting the thresholds appearing in the rule. For example, if the rule is too strict (e.g., the click-through rate threshold is too low), the conditions are dynamically relaxed. Expanding the data range covered by the rule solves the coverage blind spot problem by dynamically expanding the rule's adaptability range.

[0168] In addition, rules can be added for data that is not covered by existing rules, and this disclosure does not limit this.

[0169] Additionally, if the coverage of this rule is less than the coverage threshold and this rule is a critical rule, an alert will be triggered. Critical rules are used to monitor preset functions or preset metrics of the promotional resource integration package. For example, preset functions are core functions, such as promotional resource exposure and click-through rates; preset metrics are critical metrics, such as crash rate, click-through rate, and conversion rate. By monitoring anomalies in core functions or critical metrics in real time (such as a sudden increase in crash rate or a sharp drop in click-through rate), alerts can be quickly triggered and a repair process can be initiated.

[0170] It should be noted that, in addition to the parts mentioned above, Figure 4 The document also showcases a data visualization platform. This platform displays data used for testing, such as a centralized display of all available data, including basic data or enumeration numbers. In addition, it can display rules relating the data and the corresponding test points covered by those rules, demonstrating the mapping relationship between data and rules. Furthermore, it can display data coverage, quantifying the degree of matching between rules and data in chart form, allowing users to identify rules with low coverage.

[0171] In 302, the server controls the first terminal and the second terminal to execute the same test cases; the test cases are used to simulate user operations according to a preset operation path, and the preset operation path is related to the type of promotional resources displayed.

[0172] Since the user action path for each type of promotional resource is enumerable, standardized user action paths can be set for different types of promotional resources. Taking interstitial promotional resources as an example, the corresponding user action path is: exposure → click → redirect to landing page → close.

[0173] In addition, each step in the user's action path represents a state, forming an enumerable path chain. For the final state of the user's action path (such as the user completing a conversion or leaving the page), the following response can be set: if the user frequently returns to the ad placement from the landing page, readjust the exposure strategy.

[0174] In addition, to prevent abnormal situations, the solution also sets a timeout period, after which the promotional resources will be automatically re-exposed. For example, if the landing page times out or there is no response when clicking, a timeout judgment will be automatically triggered, and the exposure process will be automatically re-executed after the timeout.

[0175] The first point to clarify is that the initiation, flow control, and result collection of test cases are centrally managed by the server. Specifically, the server uses a UI automation framework to write test scripts. When these scripts run, they send operation commands to both online and grey-scale devices, controlling the parallel execution of the same test cases on both the online and grey-scale devices. These test cases simulate user actions according to a preset operation path. In other words, the test cases simulate real user behavior, not manual operation by a real user. Furthermore, the preset operation path is included as a parameter in the test cases. This preset operation path is adapted to the type of promotional resources displayed and includes multiple operation steps.

[0176] The following is combined Figure 5 The user operation flow and user conversion flow after the test cases begin execution are described.

[0177] See Figure 5 Once the test cases start running on the device, they directly redirect to the playback page of the promotional resource via a deep link. The exposure of the promotional resource involves the server distributing promotional resource configuration data in the form of mock data. As mentioned earlier, this configuration data includes data collected from the online production environment, which will not be elaborated upon here.

[0178] Next, simulate real user behavior, that is, simulate users clicking on conversion controls such as "Download Now" or "Claim Reward". For example, Figure 5The system uses online recording and playback to simulate user actions. Device filtering involves filtering click coordinate data collected in the online production environment based on device parameters (such as device model) to pinpoint compatibility issues (e.g., failure to load promotional resources on low-end devices). Scene filtering involves categorizing click coordinate data collected in the online production environment according to the type of promotional resource (such as banners, interstitials, or rewarded videos) to analyze conversion rate differences across different scenarios. Additionally, Figure 5 The document illustrates three conversion paths: a download page redirecting to the app store, an H5 landing page (opening a webpage link), and a DP (Deeplink Partner) launcher (launching other apps, such as payment tools, via Deeplink Partner). The BACK action refers to the user clicking the back control during the conversion process. Additionally, restarting the playback page after N seconds is used to test its resilience to failures (e.g., automatic retries after network outages).

[0179] It should be noted that random clicks may lead to a large number of invalid operations (such as clicking blank areas or non-critical controls), making it impossible to guarantee that effective actions (such as registration or download) will be triggered after a click, resulting in low click effectiveness. To address this issue, this disclosure adopts a customized UI automation approach. The core objective of customized UI automation is to improve test effectiveness while reducing maintenance costs through precise control of click behavior. Specifically, firstly, the click strategy is optimized from "completely random clicks" to "rule-guided random clicks" or "real behavior reproduction." Secondly, invalid operations are reduced, avoiding clicks on undisputed areas and focusing on critical controls. For rule-guided random clicks, the click range is narrowed down to known page elements, thus improving click effectiveness. For real behavior reproduction, the system reproduces the real user's path in the test environment, making the click path closer to the real scenario. This directly reproduces the effective conversion chain, reducing invalid clicks and further improving click effectiveness, ensuring that more effective operations are triggered within a limited time, such as guaranteeing sufficient exposure and click event reporting within a single grayscale period.

[0180] Based on the above description, test cases are used to simulate user operations according to a preset operation path during execution, including: test cases are used to perform random interactive operations (usually clicks) on valid page elements when executing operation steps; wherein, valid page elements are used to guide users to perform interactive operations.

[0181] For rule-guided random clicks, a set of clickable elements is maintained for each page. Clickable page elements are located using control IDs, image features, text content, or region coordinates. The specific implementation process includes:

[0182] Standardize naming conventions in advance, such as defining a unique set of page IDs for each page (e.g., playback page, end page, or landing page). Additionally, standardize page elements; for example, name controls on each page according to a unified set of rules. This allows for effective conversion by specifying control IDs, recognizing image features, identifying text content, and specifying area coordinates. This avoids invalid clicks, eliminates the need for frequent page structure adaptations, and reduces maintenance costs.

[0183] Based on the above description, the test cases are used to simulate user operations according to the preset operation path when they are executed. They also include: when the test cases execute operation steps, they are used to perform interactive operations according to the coordinate data collected in the online production environment.

[0184] The collected coordinate data includes at least one of the following:

[0185] Coordinate data adapted to the device, that is, coordinate data that matches the device parameters (such as device model or screen size) of the first terminal and the second terminal;

[0186] Coordinate data collected within a preset time period, such as hourly data playback, which means playing back the collected coordinate data in hourly units.

[0187] In 303, the server obtains the first execution result reported by the first terminal and the second execution result reported by the second terminal; the server compares the difference between the first execution result and the second execution result in at least one indicator, and the server outputs the test result of the version to be tested based on the difference.

[0188] In some embodiments, obtaining the first execution result reported by the first terminal and the second execution result reported by the second terminal may include, but is not limited to, the following methods:

[0189] Receive a first execution result reported by a first terminal through a data collection point embedded in the official version; wherein the first execution result carries a first type identifier, which is used to indicate that the data comes from the official version;

[0190] The system receives a second execution result reported by a second terminal through a data collection point embedded in the version under test; wherein the second execution result carries a second type identifier, which is used to indicate that the data comes from the version under test.

[0191] Data collection points, also known as technical tracking points, provide data support for inspection and maintenance by reporting data, ensuring system stability and achieving business objectives. In this embodiment, data reported by data collection points is marked with a first type of identifier and a second type of identifier, enabling the server to quickly distinguish between data from gray-scale devices and online devices, thus avoiding data contamination. Furthermore, the first and second type identifiers also allow the server to quickly filter data, thereby comparing behavioral differences between gray-scale and official versions.

[0192] Furthermore, the data reporting method using technical data tracking makes the comparison metrics more flexible. As long as the meaning of the reported fields is clearly defined, downstream data consumers can analyze the reported tracking data and create corresponding comparison reports. Simultaneously, this solution also supports customized addition of technical data tracking points. In other words, the comparison capabilities provided by this solution are highly flexible, and monitoring of some core functionalities can be added at any time. For example, monitoring of the swaying function and callback function is already supported, demonstrating strong scalability. Accordingly, the method provided in this embodiment also includes: adding fields to the embedded data collection points. These newly added fields, when executed on the first or second terminal, are responsible for collecting and reporting data used to statistically analyze the new metrics (such as the swaying trigger rate metric for a newly added swaying function).

[0193] In summary, the gray-scale testing scheme provided in this disclosure enables dual-device comparative testing of the iOS promotional resource integration package before its official release. This scheme bypasses iOS's plugin limitations, achieving testing results similar to traditional gray-scale testing schemes. Specifically, one device in the dual-device setup has a gray-scale version of the promotional resource integration package installed, while the other device has the official version. This testing mode allows the server to directly control the data input of both devices in a test environment isolated from the online production environment. By running different versions of the SDK in parallel on both devices, the differences in various metrics between the different SDK versions can be verified. By comparing the execution results of the two devices, the server can quickly locate functional and stability issues in the gray-scale version and fix them directly in the test environment without waiting for app store approval. This scheme enables the early detection of potential functional and stability issues in the new SDK version before its official release, thereby ensuring a good user experience after the new SDK version is fully launched and reducing user-side risks.

[0194] Furthermore, by capturing real data in the online production environment and supplementing it with data from the test environment, a high-quality, reusable data source is constructed. In other words, through precise online traffic capture and dynamic supplementation from the test environment, this solution builds a hybrid data pool covering both real and edge scenarios. Because the data source covers multiple scenarios, the coverage of promotional resources in scenario construction is improved, ensuring that the data is authentic, abundant, and diverse. Additionally, the data is timely (with expiration rules set) and targeted (e.g., it can be filtered based on device), avoiding unusable or invalid data interference. Moreover, because the test environment can supplement missing online scenarios (e.g., extreme network conditions, low-end device performance), the test coverage is improved. These measures significantly improve data simulation accuracy. Furthermore, by distributing realistic and diverse promotional resource configuration data, the diversity of promotional resource types and conversion behaviors can be significantly improved. Additionally, this implementation also supports the ability to distribute specified types of promotional resources according to user needs. That is, the server can obtain promotional resource configuration data from the data source that matches user needs.

[0195] Furthermore, by matching rules in the rule set with data from the data source, and performing rule count verification after matching, this solution can ultimately generate standardized test case sets and input them into the test case library. Automated rule matching reduces labor costs. The dual-mode matching setting offers flexibility and is suitable for various scenarios. Additionally, the rule count verification ensures the integrity and accuracy of the test case data, preventing dirty data from contaminating the test case library.

[0196] Furthermore, the embodiments disclosed herein generate rule sets based on a dual-track model of automated generation and manual input, which can cover both high-frequency standardized scenarios and complex customized needs. In addition, the rule base is linked to use case sets, data sources, and statistics & optimization components, ultimately using a data visualization platform to display the coverage rate of rules in the rule base, associated rules and test points, and alerts triggered by high-priority rules.

[0197] Furthermore, by statistically analyzing coverage rates, the degree of coverage between the rule set and the data source can be quantified, identifying rules not covered by data or data not covered by rules. Additionally, when the coverage rate of a rule falls below a coverage threshold, the solution will automatically perform repairs, such as data completion or rule adjustments, optimizing test coverage. Moreover, the high-priority rule alerting mechanism enables real-time monitoring of the uncovered status of key rules and timely alerts when core functions or key metrics exhibit anomalies.

[0198] Figure 6 This is a block diagram illustrating a grayscale testing apparatus for a promotional resource integration package according to an exemplary embodiment. (Refer to...) Figure 6 The device includes:

[0199] The distribution module 601 is configured to obtain promotional resource configuration data from a data source and distribute the same promotional resource configuration data to a first terminal and a second terminal; wherein, the target application installed on the first terminal has an official version of the promotional resource integration package embedded in it, and the target application installed on the second terminal has a test version of the promotional resource integration package embedded in it; the promotional resource configuration data is used to instruct the first terminal and the second terminal to display promotional resources in the same display position;

[0200] The control module 602 is configured to control the first terminal and the second terminal to execute the same test cases; wherein, when the test cases are executed, they are used to simulate user operations according to a preset operation path, and the preset operation path is related to the type of promotional resources displayed;

[0201] The acquisition module 603 is configured to acquire the first execution result reported by the first terminal and the second execution result reported by the second terminal;

[0202] Analysis module 604 is configured to compare the first execution result and the second execution result on at least one metric;

[0203] Output module 605 is configured to output the test results of the version under test based on the differences.

[0204] The gray-scale testing scheme provided in this disclosure enables dual-device comparative testing of the iOS promotional resource integration package before its official release. This scheme bypasses iOS's plugin limitations, achieving testing results similar to traditional gray-scale testing schemes. Specifically, one device in the dual-device setup has a gray-scale version of the promotional resource integration package installed, while the other device has the official version installed. This testing mode allows the server to directly control the data input of both devices in a test environment isolated from the online production environment. By running different versions of the SDK in parallel on both devices, the differences in various metrics between the SDK versions are verified. By comparing the execution results of the two devices, the server can quickly locate functional and stability issues in the gray-scale version and fix them directly in the test environment without waiting for app store approval. This scheme enables the early detection of potential functional and stability issues in the new SDK version before its official release, thereby ensuring a good user experience after the new SDK version is fully launched and reducing user-side risks.

[0205] In some embodiments, the apparatus further includes:

[0206] The construction module is configured to acquire first-type promotional resource configuration data collected in an online production environment; acquire second-type promotional resource configuration data generated in a test environment; wherein the second-type promotional resource configuration data is simulated data generated by recording user behavior in the test environment; and merge the first-type promotional resource configuration data and the second-type promotional resource configuration data to obtain the data source.

[0207] In other embodiments, the building module is configured as follows:

[0208] Obtain raw promotional resource configuration data collected in an online production environment;

[0209] Based on the first whitelist, the original promotional resource configuration data is filtered to obtain the first type of promotional resource configuration data.

[0210] In other embodiments, the building module is configured as follows:

[0211] Record user behavior in real time in the test environment to generate simulated promotion resource configuration data;

[0212] Based on the testing requirements, the simulated promotion resource configuration data is filtered for the first time.

[0213] Based on the second whitelist, the simulated promotion resource configuration data that passed the first filter is filtered a second time to obtain the second type of promotion resource configuration data.

[0214] In other embodiments, the preset operation path includes multiple operation steps; the test case, when executed, is used to simulate user operations according to the preset operation path, including:

[0215] The test cases are used to perform random interactive operations on valid page elements when executing the operation steps; wherein, the valid page elements are used to guide the object to perform interactive operations.

[0216] In other embodiments, the preset operation path includes multiple operation steps; the test case, when executed, is used to simulate user operations according to the preset operation path, including:

[0217] The test case, when executing the operation steps, is used to perform interactive operations based on coordinate data collected in the online production environment; wherein, the coordinate data includes at least one of the following:

[0218] Coordinate data matching the device parameters of the first terminal and the second terminal;

[0219] Coordinate data collected within a preset time period.

[0220] In other embodiments, the acquisition module is configured to:

[0221] The system receives the first execution result reported by the first terminal through a data collection point embedded in the official version; wherein the first execution result carries a first type identifier, which is used to indicate that the data originates from the official version.

[0222] The second execution result reported by the second terminal through the data collection point embedded in the version under test is accepted; wherein the second execution result carries a second type identifier, which is used to indicate that the data comes from the version under test.

[0223] In other embodiments, the device further includes:

[0224] The first processing module is configured to add a field to the data collection point; wherein, when the added field is executed on the first terminal or the second terminal, it is responsible for collecting and reporting data for statistical analysis of the new indicators.

[0225] In other embodiments, the device further includes:

[0226] The rule matching module is configured to match the promotion resource configuration data included in the data source with the rules included in the rule set to obtain the rule matching result;

[0227] The verification module is configured to perform rule count verification, which is used to verify whether the matching result of the verification rule conforms to the coverage of the rule declaration;

[0228] The storage module is configured to convert the validated rule matching results into test cases and store them in the test case library to form a test case set.

[0229] In other embodiments, the device further includes:

[0230] The generation module is configured to automatically generate rules for different test data types based on the test objectives and test scenarios to obtain standardized rules; obtain manually entered rules to obtain non-standardized rules; and merge the standardized rules and the non-standardized rules to obtain the rule set.

[0231] In other embodiments, the device further includes:

[0232] The statistics module is configured to calculate the coverage rate of any rule in the rule set; wherein the rule coverage rate refers to the ratio of the amount of data in the data source that actually triggers the rule to the total amount of data that may trigger the rule;

[0233] The second processing module is configured to perform a target operation if the coverage of the rule is less than a coverage threshold. The target operation includes: adjusting the rule; or expanding the data range covered by the rule.

[0234] In other embodiments, the device further includes:

[0235] The third processing module is configured to perform an alarm operation if the coverage rate of the rule is less than the coverage rate threshold and the rule is a critical rule; wherein the critical rule is used to monitor the preset functions or preset indicators of the promotion resource integration package.

[0236] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0237] It should be noted that the grayscale testing device for the promotional resource integration package provided in the above embodiments is only illustrated by the division of the above functional units. In practical applications, the above functions can be assigned to different functional units as needed, that is, the internal structure of the electronic device can be divided into different functional units to complete all or part of the functions described above. In addition, the grayscale testing device for the promotional resource integration package and the grayscale testing method embodiment of the promotional resource integration package provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiment, which will not be repeated here.

[0238] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0239] In some embodiments, when the electronic device is provided as a server Figure 7 This is a block diagram of a server according to an embodiment of the present disclosure. The server 700 can vary considerably depending on its configuration or performance, and includes one or more Central Processing Units (CPUs) 701 and one or more memories 702. The memories 702 store at least one line of program code, which is loaded and executed by the processor 701 to implement the gray-scale testing method for the promotional resource integration package provided in the various method embodiments described above. Of course, the server 700 also has wired or wireless network interfaces, a keyboard, and input / output interfaces for input / output. The server 700 also includes other components for implementing device functions, which will not be elaborated here.

[0240] In some embodiments, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, which are executed by a processor of an electronic device to implement the grayscale testing method of the aforementioned promotional resource integration package. In other embodiments, the computer-readable storage medium is a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0241] In some embodiments, a computer program product is also provided, including a computer program that, when executed by a processor of an electronic device, implements the gray-scale testing method for the promotional resource integration package described above.

[0242] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0243] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A gray-scale testing method for promoting resource integration packages, characterized in that, The method includes: The system retrieves promotional resource configuration data from a data source and sends the same promotional resource configuration data to both the first terminal and the second terminal. The first terminal has a target application with an embedded official version of the promotional resource integration package, while the second terminal has a target application with an embedded test version of the promotional resource integration package. The promotional resource configuration data is used to instruct the first terminal and the second terminal to display promotional resources in the same display position. The first terminal and the second terminal are controlled to execute the same test cases; wherein, when the test cases are executed, they are used to simulate user operations according to a preset operation path, and the preset operation path is related to the type of promotional resources displayed; Obtain the first execution result reported by the first terminal and the second execution result reported by the second terminal; Compare the first execution result and the second execution result on at least one metric, and output the test result of the version to be tested based on the difference.

2. The grayscale testing method according to claim 1, characterized in that, The method further includes: Acquire the first type of promotion resource allocation data collected in the online production environment; Obtain the second type of promotional resource configuration data generated in the test environment; wherein, the second type of promotional resource configuration data is simulated data generated by recording user behavior in the test environment; The data source is obtained by merging the first type of promotional resource configuration data with the second type of promotional resource configuration data.

3. The grayscale testing method according to claim 2, characterized in that, The acquisition of the first type of promotional resource configuration data collected in the online production environment includes: Obtain raw promotional resource configuration data collected in an online production environment; Based on the first whitelist, the original promotional resource configuration data is filtered to obtain the first type of promotional resource configuration data.

4. The grayscale testing method according to claim 2, characterized in that, The acquisition of the second type of promotional resource configuration data generated in the test environment includes: Record user behavior in real time in the test environment to generate simulated promotion resource configuration data; Based on the testing requirements, the simulated promotion resource configuration data is filtered for the first time. Based on the second whitelist, the simulated promotion resource configuration data that passed the first filter is filtered a second time to obtain the second type of promotion resource configuration data.

5. The grayscale testing method according to claim 1, characterized in that, The preset operation path includes multiple operation steps; The test cases, when executed, are used to simulate user operations according to a preset operation path, including: The test cases are used to perform random interactive operations on valid page elements when executing the operation steps; wherein, the valid page elements are used to guide the object to perform interactive operations.

6. The grayscale testing method according to claim 1, characterized in that, The preset operation path includes multiple operation steps; The test cases, when executed, are used to simulate user operations according to a preset operation path, including: The test case, when executing the operation steps, is used to perform interactive operations based on coordinate data collected in the online production environment; wherein, the coordinate data includes at least one of the following: Coordinate data matching the device parameters of the first terminal and the second terminal; Coordinate data collected within a preset time period.

7. The grayscale testing method according to claim 1, characterized in that, The step of obtaining the first execution result reported by the first terminal and the second execution result reported by the second terminal includes: The system receives the first execution result reported by the first terminal through a data collection point embedded in the official version; wherein the first execution result carries a first type identifier, which is used to indicate that the data originates from the official version. The second execution result is received by the second terminal through a data collection point embedded in the version under test; wherein the second execution result carries a second type identifier, which is used to indicate that the data comes from the version under test.

8. The grayscale testing method according to claim 7, characterized in that, The method further includes: A new field is added to the data collection point; wherein, when the new field is executed on the first terminal or the second terminal, it is responsible for collecting and reporting data for statistical analysis of the new indicators.

9. The grayscale testing method according to claim 1, characterized in that, The method further includes: The promotion resource configuration data included in the data source is matched with the rules included in the rule set to obtain the rule matching result; Perform a rule count check, which is used to verify whether the rule matching result conforms to the coverage of the rule declaration; After the rules that pass the verification are converted into test cases, they are stored in the test case library to form a test case set.

10. The grayscale testing method according to claim 9, characterized in that, The method further includes: Based on the test objectives and test scenarios, rules are automatically generated for different test data types to obtain standardized rules; Obtain manually entered rules to get non-standardized rules; The standardized rules and the non-standardized rules are combined to obtain the rule set.

11. The grayscale testing method according to claim 9, characterized in that, The method further includes: For any rule in the rule set, the coverage rate of the rule is calculated; wherein, the coverage rate of the rule refers to the ratio of the amount of data in the data source that actually triggers the rule to the total amount of data that may trigger the rule; If the coverage of the rule is less than the coverage threshold, a target operation is performed, which includes: Adjust the rules; or, Expand the data range covered by the rules.

12. The grayscale testing method according to claim 11, characterized in that, The method further includes: If the coverage of the rule is less than the coverage threshold, and the rule is a critical rule, an alarm operation is performed; The key rules are used to monitor the preset functions or preset indicators of the promotion resource integration package.

13. A grayscale testing device for promoting resource integration packages, characterized in that, The device includes: The distribution module is configured to obtain promotional resource configuration data from a data source and distribute the same promotional resource configuration data to a first terminal and a second terminal; wherein, the target application installed on the first terminal has an official version of the promotional resource integration package embedded in it, and the target application installed on the second terminal has a test version of the promotional resource integration package embedded in it; the promotional resource configuration data is used to instruct the first terminal and the second terminal to display promotional resources in the same display position; The control module is configured to control the first terminal and the second terminal to execute the same test cases; wherein, when the test cases are executed, they are used to simulate user operations according to a preset operation path, and the preset operation path is related to the type of promotional resources displayed; The acquisition module is configured to acquire the first execution result reported by the first terminal and the second execution result reported by the second terminal; The analysis module is configured to compare the first execution result and the second execution result on at least one metric; The output module is configured to output the test results of the version under test based on the differences.

14. An electronic device, characterized in that, The electronic device includes: One or more processors; Memory used to store the executable program code of the processor; The processor is configured to execute the program code to implement the grayscale testing method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the grayscale testing method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, It includes a computer program that, when executed by a processor of an electronic device, implements the grayscale testing method as described in any one of claims 1 to 12.

Images