A method and apparatus for a drop test process, a storage medium, and an electronic device

Abstract

This specification discloses a method, apparatus, storage medium, and electronic device for processing deployment tests. The method includes: monitoring service strategy experiment status data and constructing and managing a wide table of user traffic deployment experiment records; upon receiving a target strategy deployment request, obtaining a target set of test user entities, performing pre-contact conflict detection based on the wide table, and removing and isolating entities overlapping with already run test experiments; during the test run phase, periodically monitoring the wide table and the target set, performing runtime conflict detection, and triggering anomaly warnings; after the experiment, performing data overlap detection on the experimental group and the control group, and generating an experimental data independence test report based on the overlap information. This specification embodiment achieves automated conflict management throughout the entire test task lifecycle, effectively avoiding traffic pollution between concurrent experiments on the platform, and improving the independence of experimental data and the confidence of the final results.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a delivery test processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] With the rapid development of internet service platforms, data-driven strategy operation and testing (such as A / B testing) have become core methods for iterative optimization of platform services. In order to accurately evaluate the actual effects of different platform services, service platforms usually conduct a large number of high-frequency, high-concurrency strategy deployment test tasks in parallel.

[0003] However, in complex parallel testing scenarios, the target user groups of different service strategies often overlap. Existing traffic delivery management models, lacking a global dynamic verification and coordination mechanism, frequently lead to interference and contamination of experimental data between different testing tasks. This uncontrolled traffic conflict severely undermines the assumption of data independence between the experimental and control groups, significantly reducing the confidence level of the final retrieved experimental test data, thus affecting the accuracy of causal inference and the reliability of subsequent platform service deployment. Therefore, effectively monitoring and isolating user traffic conflicts in concurrent testing tasks to ensure the independence and accuracy of experimental data has become a pressing technical challenge in current platform service testing scenarios. Summary of the Invention

[0004] This specification provides a deployment test processing method, apparatus, storage medium, and electronic device, the technical solutions of which are as follows: Firstly, embodiments of this specification provide a deployment test processing method, the method comprising: In the service strategy testing scenario, monitor the service strategy experiment status data of each user traffic delivery path, and manage the wide table of user traffic delivery experiment records based on the service strategy experiment status data. When a target service strategy traffic delivery request is received, the target test user entity set selected by the target service strategy is obtained, and the test user traffic experiment conflict detection is performed on the target test user entity set according to the wide table of user traffic delivery experiment records. In the event of an experiment conflict, overlapping test user entities in the target test user entity set that overlap with the already run test experiment are removed and isolated. During the operational testing phase after the target service strategy is deployed, the wide table of user traffic deployment experiment records and the set of target test user entities corresponding to the target service strategy are periodically monitored. Based on the wide table of user traffic deployment experiment records and the set of target test user entities, deployment operation experiment conflict detection is performed to trigger an abnormal warning in the event of an experimental conflict. After the experimental testing of the target service strategy is completed, the experimental data overlap detection is performed on the experimental group and the control group of the target service strategy to obtain service test data overlap information, and an experimental data independence test report for the target service strategy is generated based on the service test data overlap information.

[0005] Secondly, embodiments of this specification provide a deployment test processing device, the device comprising: The traffic monitoring module is used to monitor the service strategy experiment status data of each user traffic delivery path in the service strategy test scenario, and manage the wide table of user traffic delivery experiment records based on the service strategy experiment status data. The delivery testing module is used to obtain the target test user entity set selected by the target service policy when it receives the target service policy traffic delivery request to be processed, and to perform experimental user traffic experiment conflict detection on the target test user entity set according to the wide table of user traffic delivery experiment records, so as to remove and isolate overlapping test user entities in the target test user entity set that overlap with the already run test experiment in the experimental conflict state. The delivery testing module is used to periodically monitor the wide table of user traffic delivery experiment records and the set of target test user entities corresponding to the target service strategy during the operation testing phase after the target service strategy is delivered. Based on the wide table of user traffic delivery experiment records and the set of target test user entities, the module performs delivery operation experiment conflict detection to trigger an abnormal warning in the event of an experimental conflict. The deployment testing module is used to perform experimental data overlap detection on the experimental group and control group of the target service strategy after the experimental test of the target service strategy is completed, obtain service test data overlap information, and generate an experimental data independence test report for the target service strategy based on the service test data overlap information.

[0006] Thirdly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0007] Fourthly, this specification provides a computer program product storing at least one instruction adapted to be loaded by a processor and to execute the method steps of one or more embodiments of this specification.

[0008] Fifthly, this specification provides a computer program product storing at least one instruction adapted to be loaded by a processor and to execute the method steps of one or more embodiments of this specification.

[0009] Fifthly, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0010] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following: In one or more embodiments of this specification, the technical problem of severe overlap of user traffic among multiple parallel test tasks in complex, high-concurrency service platform testing scenarios due to a lack of global coordination is addressed. This leads to mutual contamination of experimental variables, violation of the assumption of data distribution independence, and extremely low confidence in the final business causal inference conclusions. By constructing a wide table of experimental records spanning the entire business chain and combining it with a linkage mechanism of pre-test isolation, in-test warning, and post-test verification, closed-loop automated conflict management is achieved throughout the entire lifecycle of high-concurrency test tasks. Pre-test precise elimination and isolation based on the underlying wide table effectively prevents physical traffic contamination during the test initiation phase, ensuring the absolute purity of the initial test samples. The in-test multi-dimensional periodic dynamic monitoring mechanism breaks the limitation of traditional strategies being in a black box state after deployment, enabling agile anomaly interception and disaster recovery for sudden external task overlaps. Post-test, based on rigorous inter-group overlap deviation analysis, invisible external environmental interference is transformed into quantifiable and visualized confidence indicators of data distribution independence. This avoids variable coupling between multiple concurrent strategies, improves the overall traffic reuse efficiency of the platform while ensuring experimental orthogonality, and ensures the absolute unbiasedness of the test results of each strategy, thus providing a reference for scientific causal inference and high-risk service decision-making in upper-layer service development. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating a deployment test processing method provided in the embodiments of this specification; Figure 2 This is a schematic diagram of an experimental data overlap detection process provided in the embodiments of this specification; Figure 3 This is a schematic diagram of the structure of a delivery testing and processing device provided in the embodiments of this specification; Figure 4 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification. Detailed Implementation

[0013] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0014] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0015] The present specification will now be described in detail with reference to specific embodiments.

[0016] In one embodiment, such as Figure 1 As shown, a deployment test processing method is proposed. This method can be implemented using a computer program and can run on a deployment test processing device based on the von Neumann architecture. This computer program can be integrated into applications or run as a standalone utility application. The deployment test processing device can be a terminal device, including but not limited to: personal computers, tablets, handheld devices, vehicle-mounted devices, wearable devices, computing devices, or other processing devices connected to a wireless modem. In different networks, terminal devices can be called by different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user equipment, cellular phone, cordless phone, terminal device in 5G networks or future evolved networks, etc.

[0017] Specifically, the deployment test processing method includes: S102: In the service strategy test scenario, monitor the service strategy experiment status data of each user traffic delivery path, and manage the wide table of user traffic delivery experiment records based on the service strategy experiment status data. Service strategy testing scenarios refer to business service environments in internet platforms or digital service systems where, in order to verify and test the actual effects of different business service strategies (such as interface interaction redesign, marketing benefits distribution, recommendation algorithm adjustments, etc.), the overall user traffic is divided into different groups (such as experimental groups and control groups) for comparative testing.

[0018] User flow delivery path refers to the specific business link or endpoint where the tested user entity generates interactive behavior on the application side and triggers the policy issuance (e.g., APP homepage pop-up path, payment completion page recommendation path, message push channel, etc.).

[0019] Service strategy experiment status data refers to the data set that records whether a test experiment and the specific group were hit when the test user entity triggered the above-mentioned delivery path. Optionally, this data usually includes, but is not limited to: user unique identifier, timestamp, test task code, group identifier (experimental group / control group), and touch point source.

[0020] The wide table for user traffic delivery experiment records refers to a global, multi-dimensional data view built within the underlying data storage architecture. It aggregates and integrates experimental status data scattered across various delivery paths. For example, this wide table typically uses "user-time-experiment task" as a composite primary key to break down data silos between systems and fully record the entire lifecycle test trajectory of each test user entity.

[0021] This example illustrates the placement of tracking points on the business servers along various user traffic delivery paths. When a user triggers a service policy, the business server asynchronously records the resulting service policy experiment status data in log files. The underlying data integration engine (such as an offline synchronization tool) is then invoked to periodically retrieve the raw status logs from each business service system according to a preset time period (e.g., hourly or daily at midnight). Data processing nodes then perform data cleaning (e.g., removing invalid requests, completing missing dimensions), format standardization, and the generation of composite primary keys on the raw logs through an ETL (Extract, Transform, Load) process. Finally, the structured data is batch-written into a distributed relational data warehouse (such as Hive or ClickHouse) to update the wide table of user traffic delivery experiment records in an append or overwrite manner.

[0022] In one feasible implementation, the wide table for managing user traffic delivery experiment records based on the service policy experiment status data can be executed in the following manner: The test traffic is parsed on the service strategy experiment status data. A wide table of user traffic delivery experiment records is constructed and updated based on user attribute identifiers, date identifiers, and service experiment test task identifiers. The wide table of user traffic delivery experiment records is used to record all service test path trajectories participated in by each test user entity within the test time period.

[0023] Test traffic parsing can be understood as a data processing process that uses computer programs to extract fields, convert formats, and clean the raw, unstructured, or semi-structured service strategy experiment status data (such as front-end event logs and back-end interface request records) to obtain structured key parameters.

[0024] The service test path trajectory refers to a slice of an entity's behavior within the platform, characterized by the three identifiers mentioned above (user, date, and task). It reflects which parallel test tasks hit a specific user within a specific time period.

[0025] As an illustration, the data parsing engine reads the raw status logs and removes erroneous or redundant entries. Using regular expressions and preset JSON extraction rules, it precisely extracts user attribute identifiers, date identifiers, and service experiment test task identifiers from each log entry. A wide table for user traffic delivery experiment records is defined in a distributed columnar database. The user attribute identifier and date identifier are used as a composite primary key, and the service experiment test task identifier is used as a column family or array type field. When new data is parsed, an Upsert operation is performed: if the composite primary key does not exist in the wide table, a new row is created; if it already exists, the new test task identifier is appended to the task list field corresponding to that primary key, thus connecting the entire service test path trajectory for that user on that day within a single data row.

[0026] S104: When a target service strategy traffic delivery request to be processed is received, the target test user entity set selected by the target service strategy is obtained, and the test user traffic experiment conflict detection is performed on the target test user entity set according to the wide table of user traffic delivery experiment records, so as to remove and isolate overlapping test user entities in the target test user entity set that overlap with the already run test experiment in the state of experiment conflict. The target service strategy traffic delivery request refers to the trigger instruction (such as an API call request or a scheduled task) sent by the scheduling center or business request node within the service platform to the traffic distribution system or experimental platform in order to start a brand new test task (i.e., the target service strategy).

[0027] The target test user entity set refers to the set of test user identities initially selected based on the set initial filtering conditions (such as specific region, specific activity tags, etc.) for the newly launched test task, i.e. the initial screening population package without conflict filtering.

[0028] Experimental user traffic conflict detection refers to the technical process by which a computer system uses underlying algorithms to compare the initially screened user group with the recorded existing active test task user group in order to identify whether there are physical overlaps or collisions between individuals.

[0029] Elimination and isolation refers to the process of using computer methods such as physical deletion or logical blocking to prevent the entity from being issued the current target service policy after identifying the existence of entity crossover (experimental conflict state), thereby ensuring absolute mutual exclusion and orthogonality of traffic allocation between each test task.

[0030] In one feasible implementation, a static data comparison and physical elimination mechanism based on set operations can be adopted. After receiving a deployment request, the service platform's task scheduling system first reads the target test user entity set from the distributed file system or object storage, extracts all test user entities currently in running test experiments from the wide table of user traffic deployment experiment records, and constructs a global active entity set. The underlying system calls a set intersection algorithm (e.g., hash table-based intersection operation) to compare the target test user entity set with the global active entity set. If the calculated intersection is not empty, it is determined that an experimental conflict state has been entered. At this time, based on the generated intersection data, a set difference operation is performed in memory or a temporary database, that is, the intersection part is subtracted from the target set, thereby physically removing overlapping test user entities from the initial screening data. Finally, the target service policy is only issued to the pure entity set remaining after the set difference operation.

[0031] S106: During the operation and testing phase after the target service strategy is deployed, the wide table of user traffic deployment experiment records and the set of target test user entities corresponding to the target service strategy are periodically monitored. Based on the wide table of user traffic deployment experiment records and the set of target test user entities, the deployment operation experiment conflict detection is performed to trigger an abnormal warning in the experimental conflict state. The operational testing phase refers to the entire lifecycle period during which the target service strategy has been officially launched for testing, and the service platform is continuously distributing the strategy to the target set of test users (i.e., it is in a state of real traffic distribution).

[0032] Conflict detection in deployment and operation refers to the process where, during the operation of an experiment, due to dynamic changes in the population rules of other parallel experiments or drift in user attributes, a participant who was initially deemed "pure" before deployment (i.e., in step S104) unexpectedly crosses physical boundaries with other newly launched test experiments midway through the experiment. This detection quantifies and identifies this dynamic overlap phenomenon during the experiment.

[0033] Anomaly warning refers to the triggering mechanism that automatically generates a notification message containing anomaly attribution information when the system detects that the proportion of the above-mentioned dynamic overlap exceeds the preset safety tolerance limit, and distributes it to relevant stakeholder terminals or upper-level disaster recovery systems through the communication bus.

[0034] As an illustration, the underlying distributed task scheduling framework (such as a timer engine based on time expressions) automatically wakes up the monitoring script during the daily low-traffic period in the early morning. The monitoring script reads the latest daily partition data from the fully updated wide table of user traffic delivery experiment records and performs a cross-table join operation on the target user entity set currently in the running test phase. The service platform's system calculates the number of entities in the current target user entity set that also have the identifier of other running test experiments, and divides it by the total number of the target set to obtain the dynamic overlap ratio. Subsequently, the system compares this dynamic overlap ratio with a preset safety threshold. If it is determined that the safety threshold is exceeded, the system is confirmed to have entered an experiment conflict state. The early warning module will immediately generate a structured early warning message containing the conflict experiment identifier, the scale of contaminated entities, and the overlap trend, and push it to the terminal device of the corresponding test task manager through an asynchronous message queue, or broadcast it to the entire network through email and SMS gateways.

[0035] S108: After the experimental test of the target service strategy is completed, the experimental data overlap detection is performed on the experimental group and the control group of the target service strategy to obtain service test data overlap information, and an experimental data independence test report for the target service strategy is generated based on the service test data overlap information.

[0036] Experimental test ends: This refers to the time point when the target service policy reaches the pre-set lifecycle end point, the system automatically freezes the policy issuance actions of the corresponding set of test user entities, and stops collecting incremental business result data for this batch of tests.

[0037] The experimental group and the control group refer to two or more benchmark comparison subsets that are strictly mutually exclusively divided into the target subject user entity set in the target service strategy in order to verify the effect of a certain variable (such as a new UI or a new algorithm).

[0038] Experimental data overlap detection refers to the process by which a computer system, in the post-experiment phase, backtracks to wide table data, calculates the proportion of contamination or infiltration from other external tests conducted during the testing period for both the experimental and control groups, and quantifies whether this contamination exhibits a symmetrical distribution between the two groups.

[0039] Service test data overlap information refers to a quantitative indicator (such as inter-group distribution deviation or independence decay confidence value) derived from the above detection algorithm, characterizing the degree of asymmetry in external interference. This information is the core basis for determining whether the current test data has failed due to external cross-interference.

[0040] The experimental data independence test report is a system-level diagnostic document or visual data dashboard automatically generated by the system based on overlapping information. It is used to declare whether the distribution of experimental data for this target service strategy satisfies the statistical orthogonality hypothesis and whether the causal inference conclusions have high confidence.

[0041] In a schematic manner, after the experimental test, the service platform's system uses a data analysis engine to extract user details for both the experimental and control groups of the target service strategy, and performs correlation calculations with each run test experiment that intersects with the data in the wide table. Target quantitative indicators are calculated separately, extracting multiple first overlap percentages (i.e., the proportion of overlapping entities in the total number of experimental groups) for each run test experiment in the experimental group, and multiple second overlap percentages (i.e., the proportion of overlapping entities in the total number of control groups) for the control group. The first and second overlap percentages corresponding to the same run test experiment are subtracted, and the sum of the absolute values ​​of these differences is used as the data distribution independence deviation value (i.e., service test data overlap information). Finally, this deviation value is compared with a system-preset static independence judgment threshold (e.g., 5%). When the deviation value is determined to be less than the preset threshold, it indicates that external interference is evenly distributed in the experimental and control groups, without introducing systematic bias. A data independence test report is then generated, indicating that the data distribution independence condition is met, and a conclusion that the test results are reliable is output.

[0042] This specification addresses the technical problem in complex, high-concurrency service platform testing scenarios where multiple parallel test tasks suffer from severe overlap of user traffic due to a lack of global coordination. This overlap leads to mutual contamination of experimental variables, violation of the assumption of data distribution independence, and extremely low confidence in the final business causal inference conclusions. By constructing a wide experimental record table spanning the entire business chain and combining it with a linkage mechanism of pre-test isolation, in-test warning, and post-test verification, closed-loop automated conflict management is achieved throughout the entire lifecycle of high-concurrency test tasks. Pre-test precise elimination and isolation based on the underlying wide table effectively prevents physical traffic contamination during the test initiation phase, ensuring the absolute purity of the initial test samples. The in-test multi-dimensional periodic dynamic monitoring mechanism breaks the limitation of traditional strategies being in a black box state after deployment, enabling agile anomaly interception and disaster recovery for sudden external task overlaps. Post-test, based on rigorous inter-group overlap deviation analysis, invisible external environmental interference is transformed into quantifiable and visualized data distribution independence confidence indicators. This avoids variable coupling between multiple concurrent strategies, improves the overall traffic reuse efficiency of the platform while ensuring experimental orthogonality, and ensures the absolute unbiasedness of the test results of each strategy, thus providing a reference for scientific causal inference and high-risk service decision-making in upper-layer business.

[0043] In one feasible implementation, the specific execution of conflict detection for the deployment experiment based on the wide table of user traffic deployment experiment records and the target set of subject users, in order to trigger an anomaly warning in the event of an experimental conflict, can be carried out in the following manner: S202: Based on the wide table of user traffic delivery experiment records and the target test user entity set, determine the test run experiment overlap information between the target test user entity set and the test entity set of the run experiment; S204: Detect the experimental conflict state during the deployment operation based on the experimental overlap information of the test run, so as to trigger an abnormal warning in the experimental conflict state.

[0044] Test run overlap information refers to the data set obtained through underlying data comparison during the lifecycle of the target service strategy, representing the physical intersection of the target set of test users with other groups of test tasks running in parallel. This information typically includes, in its data structure, a detailed list of overlapping user entities, an association matrix of overlapping task identifiers, the number of newly added overlapping experiments, and quantitative indicators such as dynamic overlap ratios across different dimensions.

[0045] The experimental conflict state during deployment refers to a high-risk operational state determined by algorithms or rules based on the aforementioned overlapping information. When the target service strategy is subjected to traffic penetration from external testing tasks that exceeds the preset security tolerance, and this is sufficient to cause serious distortion or attribution fallacies in the business performance data (such as conversion rate and retention rate) collected in the current experiment, it is marked as being in this state.

[0046] Schematic, in step S202, the service platform constructs a sliding time window in memory using a stream processing framework (such as Flink), consumes user traffic logs reported by the business gateway in real time, and combines this with a wide table of user traffic delivery experiment records in memory. Using a cardinality estimation algorithm (such as HyperLogLog), it calculates the estimated instantaneous intersection of the target user entity set and each already run experiment entity set within the current time window with extremely low memory overhead, thereby outputting a real-time test run experiment overlap information stream with time-series characteristics. After proceeding to step S204, the service platform does not rely on a single static threshold but instead inputs this overlap information stream into an unsupervised anomaly detection algorithm model (such as an isolated forest or a dynamic baseline deviation model based on historical moving averages) for real-time evaluation. When the model detects a sharp increase in the instantaneous overlap ratio curve that does not conform to business patterns, or detects a sudden increase in newly added overlapping experiment tests (i.e., experiments that were originally non-intersecting suddenly exhibit large-scale crossover), it confirms an experiment conflict state within milliseconds. To achieve the highest level of protection, the service platform system automatically issues a traffic circuit breaker command through the service governance gateway when triggering anomaly warnings, thereby downgrading the policy of the contaminated target service in real time and preventing dirty data from further contaminating the underlying core evaluation indicators.

[0047] Optionally, the specific implementation of detecting experimental conflict states during deployment based on the test run overlap information can refer to the following methods: Step A2: Detect newly added overlapping experimental tests based on the test run experiment overlap information, and determine the dynamic overlap ratio between the target subject user entity set and the already run experiment subject entity set based on the test run experiment overlap information; New overlapping experimental tests refer to external test tasks that did not overlap with the target service strategy in the early stages of its operation, but were detected by the underlying wide table for the first time within the current monitoring period as physically overlapping with the target user entity set. This situation is usually caused by other business lines urgently launching new strategies midway through the process or arbitrarily expanding the selected population.

[0048] The dynamic overlap ratio refers to the ratio of the number of entities in the target subject user entity set that simultaneously hit a specific run experiment within a specific monitoring period to the total number of entities in that target subject user entity set. This ratio is a quantitative characteristic that fluctuates over time.

[0049] Step A4: If there is a newly added overlapping experimental test and / or the dynamic overlap ratio is greater than the preset safety threshold, it is determined to be an experimental conflict state, and test warning information including conflict entity details is generated and the test warning information is distributed to the test task terminal.

[0050] The preset safety threshold refers to the maximum crossover ratio of traffic that the service platform can tolerate (e.g., 3% or 5%). When the overlap ratio is below this threshold, external interference is statistically considered to be within the range of controllable white noise; once it exceeds this threshold, orthogonality is considered to be destroyed.

[0051] Conflict Entity Details: This refers to the data granularity information of the specific subjects whose data overlaps or intersects. It typically includes a list of unique identifiers (UIDs) of the underlying users who are in conflict, the timestamp that triggered the conflict, the distribution of conflict characteristics, and the corresponding experimental code of the contamination source.

[0052] In one feasible implementation, during step A2, the service platform reads the test run overlap information generated in the current cycle through the rule judgment engine and extracts the overlap experiment identifier list. The system performs a difference operation between this and the historical overlap experiment identifier list cached in the previous monitoring cycle. If the difference is not empty, the task corresponding to the difference is accurately identified as a newly added overlap experiment test. Simultaneously, the dynamic overlap ratio corresponding to each overlap task is calculated using an aggregation function. Then, step A4 is executed, where the rule engine performs a judgment based on a set logical OR condition: if a newly added overlap experiment test is detected, or if the dynamic overlap ratio of any existing overlap task exceeds the statically configured preset safety threshold, the system immediately switches its state machine to the experiment conflict state. Subsequently, the system's underlying batch processing task, based on the overlap association key, reverse-associates and extracts the specific conflict entity details from the data warehouse and renders them into a preset test warning information HTML template. Finally, the service platform calls the enterprise's internal message gateway interface to accurately distribute the warning message containing detailed attachments to the test task terminal or R&D manager's workbench in the form of an encrypted email or asynchronous work order.

[0053] In the embodiments of this specification, by introducing a dual judgment mechanism that detects newly added overlapping experiments and monitors the dynamic overlap ratio, the technical problem of test data being hidden and contaminated due to sudden external task overlap or a sudden increase in existing overlap during strategy operation is effectively solved. This design can not only capture unexpected traffic intrusion and ratio exceeding risks in real time with extremely high sensitivity, ensuring accurate identification of experimental conflict status at the first moment, but also provide business terminals with references with direct operational guidance value by automatically generating and distributing early warning messages containing details of specific conflict entities. This empowers testers to intervene agilely and remove and clean up test data in the early stages of irreversible damage, thus maximizing the preservation of the purity of core evaluation indicators and the scientific nature of the final business decision.

[0054] Optional, please refer to Figure 2 , Figure 2 This is a flowchart illustrating the process of experimental data overlap detection. Specifically, the experimental group and control group of the target service strategy are subjected to experimental data overlap detection to obtain service test data overlap information. The following method can be used as a reference: S302: Calculate the first experimental orthogonal quantitative index corresponding to the target experimental group and the second control orthogonal quantitative index corresponding to the target control group of the target service strategy, respectively. The first experimental orthogonal quantification index includes the first overlap ratio corresponding to each run test experiment, the first overlap ratio representing the overlap ratio of the subject user entity group corresponding to the run test experiment in the total number of target experimental group users; the second control orthogonal quantification index includes the second overlap ratio corresponding to each run test experiment, the second overlap ratio representing the overlap ratio of the subject user entity group corresponding to the run test experiment in the total number of target control group users. Target experimental group / target control group: refers to the baseline comparison subset strictly isolated and segmented by the traffic routing gateway (such as a hash modulo algorithm based on user ID) when the target service policy is issued. Typically, the experimental group is subject to new policy variables, while the control group maintains its original business baseline.

[0055] First experimental orthogonal quantification index / Second control orthogonal quantification index: These are mathematical vectors or multidimensional feature arrays derived by the service platform's computer system through underlying correlation statistical calculations, used to describe the degree of penetration of the experimental group and the control group by global external parallel testing tasks, respectively.

[0056] The first overlap ratio / second overlap ratio refers to the specific numerical elements that constitute the above orthogonal quantification index. In a physical sense, its calculation logic is a precise ratio: the numerator is the absolute number of entities simultaneously in a specific external running test experiment and the current target experimental subgroup (experimental group or control group), and the denominator is the total number of test entities in the target experimental subgroup.

[0057] To illustrate, the service platform invokes the underlying distributed computing engine (such as a Hive or Spark cluster) to extract the full lifecycle snapshot data of the target service strategy from the persistently stored wide table of user traffic delivery experiment records. Based on group identifiers, the user details are strictly divided into target experimental group data clusters and target control group data clusters. The computing engine triggers a join query task, performing a full table traversal and comparison between these two data clusters and the set of users in the wide table who are currently active and have run test experiments. The service platform uses aggregation operators to accurately count the total number of user entities in the experimental and control groups that match each external experiment after deduplication. The service platform executes a division instruction, dividing the total number of overlapping entities in each external experiment by the total sample size of the target experimental group and the target control group, respectively, to obtain a series of floating-point ratio values. Finally, the ratio values ​​for the corresponding experimental group are encapsulated and concatenated into an array structure to generate the first experimental orthogonal quantification index; similarly, the ratio values ​​for the corresponding control group are encapsulated to generate the second control orthogonal quantification index.

[0058] S304: Based on the first experimental orthogonal quantization index and the second control orthogonal quantization index, perform data distribution independence condition analysis to obtain the service test data overlap information.

[0059] Data distribution independence condition analysis refers to the technical process by which the underlying data analysis engine or rule engine of a computer system aligns, compares, and calculates the difference between two sets of quantitative indicators (i.e., the first experimental orthogonal quantitative indicator and the second control orthogonal quantitative indicator) representing the degree of interference in the experimental and control groups. Essentially, it infers whether the current test data satisfies the statistical orthogonality hypothesis by analyzing the symmetry of the distribution of external cross-flows in the two groups of the target strategy.

[0060] Service test data overlap information refers to the final quantitative result or status identifier calculated by the above-mentioned parsing algorithm. This information is usually expressed as a specific deviation value (such as the inter-group distribution deviation value), a statistical confidence score, or a Boolean status label that directly represents whether the data distribution independence condition is met or not. It is the core data entity for generating the final test report.

[0061] In one feasible implementation, during this step, the underlying data parsing engine of the service platform extracts the first experimental orthogonal quantization index (containing multiple first overlap ratios) and the second control orthogonal quantization index (containing multiple second overlap ratios) calculated in the previous step. The overlap ratios with the same external running test experiment identifier are aligned. Subsequently, the service platform's parsing engine calls the underlying subtraction operator to calculate the difference between the first and second overlap ratios corresponding to the same experiment identifier, and takes their absolute values. Then, by using an accumulation register, the absolute values ​​of the differences of all aligned experiments are summed to obtain a macroscopic inter-group distribution deviation value. Finally, this inter-group distribution deviation value is compared with a preset static independence judgment threshold (e.g., a tolerance of 5%). When the inter-group distribution deviation value is determined to be less than the preset threshold, the system determines that the external interference traffic is evenly distributed in the experimental and control groups, and then generates and outputs service test data overlap information in memory that represents the data distribution independence condition; otherwise, it generates overlap information that represents the data distribution independence condition, indicating that the current test data has failed due to severe asymmetric cross-contamination.

[0062] In one feasible implementation, specifically performing the data distribution independence condition analysis based on the first experimental orthogonal quantification index and the second control orthogonal quantification index to obtain the service test data overlap information can be done in the following manner: 1) Extract the total number of run test experiments that overlap with the target service strategy; The total number of run test experiments refers to the absolute number of other parallel test tasks retrieved from the underlying wide table within the testing period of the target service strategy, which actually generated physical traffic intersections or overlaps with the target set of test user entities. This value constitutes the loop boundary for subsequent summation operations.

[0063] 2) Based on the first experimental orthogonal quantification index and the second control orthogonal quantification index, the data distribution independence deviation value is calculated using the first calculation formula. When the data distribution independence deviation value is determined to be less than the preset independence judgment threshold, service test data overlap information representing that the data distribution independence condition is met is generated. When the data distribution independence deviation value is determined to be greater than or equal to the preset independence judgment threshold, service test data overlap information representing that the data distribution independence condition is not met is generated. The first calculation formula satisfies the following formula:

[0064] Wherein, Diff represents the data distribution independence deviation value, which refers to the absolute number of other parallel test tasks retrieved through the underlying wide table within the test period of the target service strategy that actually generated physical traffic intersections or overlaps with the target test user entity set. This value constitutes the loop boundary for subsequent summation operations.

[0065] N represents the total number of test experiments that have been run, Ti represents the first overlap ratio, Ci represents the second overlap ratio, and i is a positive integer representing the label of the test experiment that has been run.

[0066] The independence threshold is a pre-configured hard boundary value used to determine whether test data has become invalid due to external interference. This threshold is typically set by the system administrator based on the service platform's historical A / B test fault tolerance rate or statistical confidence level (such as 5% or 3%).

[0067] To illustrate, a query request is first initiated to extract the total number N of run test experiments that physically overlap with the target service strategy from the service test data overlap information set. The parsing service initializes an accumulator register in memory with an initial value of zero to store the data distribution independence deviation value Diff to be calculated. The underlying layer performs a loop of N iterations. In each loop (i.e., for the i-th run test experiment), the corresponding first overlap ratio Ti and second overlap ratio Ci are extracted, a floating-point unit is called to execute a subtraction instruction, and the absolute value function is applied to the difference. The absolute difference calculated in a single loop is accumulated in real time into the aforementioned register. After N loops are completed, the final value output in the register is the Diff value of the target strategy. This Diff value is input into a comparator and compared with a preset independence judgment threshold in memory. When the comparator determines that the deviation value is less than the preset threshold, the system state machine triggers the "pass" branch, automatically generating and persisting a service test data overlap information tag that represents "satisfying the data distribution independence condition"; conversely, if the comparator determines that the deviation value is greater than or equal to the preset threshold, the system triggers the "intercept" branch, generating alarm overlap information that represents "not meeting the condition".

[0068] In one feasible implementation, specifically performing the data distribution independence condition analysis based on the first experimental orthogonal quantification index and the second control orthogonal quantification index to obtain the service test data overlap information can be done in the following manner: 1) Extract the total number of user entities for experimental testing of the target service strategy; 2) Based on the data calculated by the first experimental orthogonal quantization index and the second control orthogonal quantization index, the data distribution independence decay confidence value is calculated using the second calculation formula. When the data distribution independence decay confidence value is determined to be less than the preset independence judgment threshold, service test data overlap information representing that the data distribution independence condition is met is generated. When the data distribution independence decay confidence value is determined to be greater than or equal to the preset independence judgment threshold, service test data overlap information representing that the data distribution independence condition is not met is generated. The second calculation formula satisfies the following formula:

[0069] Wherein, Conf is the confidence value for the independence of the data distribution, K represents the total number of user entities in the experimental test, Ti represents the first overlap ratio, Ci represents the second overlap ratio, and i is a positive integer and represents the label of the test experiment that has been run.

[0070] For illustrative purposes, the total number of user entities N in the experimental test refers to the overall user sample size participating in the current target service strategy test (i.e., the sum of the number of user entities in the target experimental group and the target control group).

[0071] The data distribution independence decay confidence value (Conf) refers to the final evaluation quantification index output by the system's underlying computing engine through a specific nonlinear second calculation formula of this invention. This index integrates features such as sample cardinality penalty, multiple concurrency decay, and nonlinear variance amplification, and is a comprehensive confidence score that can accurately measure small systematic biases under massive data.

[0072] Optionally, the process of removing and isolating overlapping test user entities in the target test user entity set that overlap with the already run test experiments can also be performed in the following ways: S402: Identify overlapping test user entities in the target test user entity set that overlap with the run test experiment, extract the target multidimensional attribute features corresponding to the target test user entity set, and determine the initial baseline feature distribution of the target test user entity set based on the target multidimensional attribute features; Overlapping test subject entities refer to the set of user identifiers that have been assigned to other parallel active test tasks at the current time point in the target service policy test subject population initially screened by the system, and would cause cross-contamination if a new policy is issued at the same time.

[0073] Target multidimensional attribute features refer to a set of vectorized labels used to create a multidimensional and comprehensive business profile of an individual user in a computer system. These features are typically abstracted from the underlying data platform and include, but are not limited to, high-dimensional sparse or dense feature variables such as basic demographic attributes, platform activity levels, historical consumption preferences, and device terminal models.

[0074] The initial baseline feature distribution refers to the statistical probability density, frequency histogram, and probability quality function of the entire target user entity set on various target multidimensional attribute features in its original state, without any removal or physical isolation of the aforementioned overlapping user entities. This distribution represents the broad audience that this strategy aims to reach and serves as a benchmark for subsequently measuring the degree of data distortion.

[0075] In a schematic manner, a distributed association query is first initiated on the wide table of user traffic delivery records in persistent storage to compare the intersection of the target subject user entity set with the entities of each running test experiment, thereby logically identifying the overlapping subject user entities. Then, a feature engineering computing cluster (such as a node array based on Apache Spark MLlib) is invoked to start a data extraction task. Using the unified identity identifier of the users in the initial screening population as the primary key, the underlying user profile wide table or feature storage service is called across the cluster to pull the feature vectors of all users in the target subject user entity set into the memory space of the elastic distributed dataset and data frame, completing the extraction of target multi-dimensional attribute features. The computing nodes call aggregation operators to perform full statistical summarization for each key attribute dimension, calculate the proportion of each feature enumeration value in the total target population, serialize these proportion data into a series of feature probability quality functions, and persist them as the initial baseline feature distribution of the target subject user entity set to the high-speed cache center for subsequent relative entropy verification services to pull and compare at any time.

[0076] S404: Perform a virtual culling operation on the overlapping subject user entities in memory space and intercept physical deletion instructions targeting the underlying database; Virtual culling refers to the logical masking, filtering, or state bit flipping of data nodes marked with overlapping attributes in the cache of a computer system or the context view of an application, so that this part of the data disappears from the subsequent visible view of the current operation cycle, rather than actually erasing the original data body.

[0077] Physical deletion commands refer to control commands (such as the DELETE statement or DROP command in a relational database) sent by a business system or data processing component to the underlying persistent storage engine (such as MySQL, HBase, etc.) to permanently erase data records from physical disk sectors.

[0078] Interception can be a computer security control action that captures, suspends, blocks, or rewrites the aforementioned physical operation instructions before they actually reach the underlying storage engine and are executed, through gateway proxies, middleware interceptors, and database transaction isolation mechanisms.

[0079] S406: Based on the target multidimensional attribute features, re-statistically analyze the feature distribution of the remaining test user entities after virtually removing the overlapping test user entities, and obtain the remaining test feature distribution; Remaining test user entities: refers to the set of pure user entities that remain after the virtual culling operation in step S404 in the memory work area or the isolated transaction view of the database, and that have not participated in any other conflict experiments.

[0080] Re-statistics refers to the process by which the underlying computing engine of a computer system, based on new data boundaries (i.e., the aforementioned residual set), triggers mathematical operations for aggregation, classification, and frequency statistics on specific dimensions. Its purpose is to obtain the latest macroscopic form of the data after filtering.

[0081] The remaining subject feature distribution refers to the novel probability quality function and frequency distribution histogram calculated for the aforementioned pure user set across pre-defined target multi-dimensional attribute feature dimensions. This distribution accurately reflects the profile of the population that would ultimately enter the target service strategy test if physical deletion were implemented.

[0082] Indicatively, the underlying distributed computing engine (such as an Apache Spark cluster) first reads the memory snapshot view (e.g., a filtered DataFrame) generated in the previous step, which only contains the remaining test user entities. Then, the computing nodes call the underlying feature aggregation operator to strictly align with the target multi-dimensional attribute feature list established in step S402, performing multi-threaded concurrent scanning and classification statistics on the massive user feature vectors in the memory view. The absolute number of this remaining group under each feature dimension enumeration value (e.g., different activity levels, different consumption frequency intervals) is accurately calculated and uniformly divided by the total number of remaining users for normalization, generating the probability proportion of each feature enumeration value. These multi-dimensional proportion data are serialized and concatenated into a high-dimensional probability vector, forming a remaining test user feature distribution that can completely depict the overall profile after removing overlapping users. This distribution is permanently stored in a distributed cache cluster, providing standardized input for downstream relative entropy comparison operations.

[0083] S408: Calculate the relative entropy between the remaining subject feature distribution and the initial baseline feature distribution to obtain the feature distribution offset; Relative entropy, also known as KL divergence, is a mathematical metric used to accurately measure the asymmetric difference between two probability distributions. In this invention, it is innovatively applied to quantify the similarity between two business profile distributions; the smaller the value, the closer the two distributions are. In practical applications, the remaining subject feature distribution and the initial baseline feature distribution are input into the relative entropy calculation model to calculate the offset of the output feature distribution. Optionally, an initial baseline feature distribution P and the remaining subject feature distribution Q are determined. For each dimension of the target multidimensional attribute features (e.g., age, user service level, activity level, etc.), it is pre-divided into k mutually exclusive feature intervals, where k can be understood as the total feature dimension of the target multidimensional attribute features; the initial baseline feature distribution P is composed of P(x) of each feature dimension. i Composed of P(x) i P(x) represents the probability that a user falls into the i-th feature interval under the initial baseline state. i The initial baseline feature distribution P is composed of Q(x) for each feature dimension; the remaining subject feature distribution Q is composed of Q(x) for each feature dimension. i Composed of ) Q(x i ) represents the probability that a user falls into the i-th feature interval in the remaining states after virtual elimination, and all Q(x) i The remaining subject characteristic distribution Q is formed by all P(x). i The sum of all Q(x) is 1. i The sum of these is 1. That is, after determining the initial baseline feature distribution P and the remaining subject feature distribution Q, the feature distribution offset D can be calculated according to the following relative entropy calculation model. KL :

[0084] Feature distribution offset: refers to the specific scalar value or tensor result output by the computer system after executing the aforementioned relative entropy algorithm. This indicator objectively and accurately reflects the degree to which the final remaining population sample profile deviates from the platform's initial overall profile after the aforementioned virtual elimination operation is performed.

[0085] S410: Determine whether the feature distribution offset is greater than or equal to a preset system tolerance threshold. If the feature distribution offset is less than the system tolerance threshold, then remove and isolate overlapping test user entities.

[0086] This specification introduces a memory-level virtual sandbox simulation and physical database instruction interception middleware into the underlying architecture, providing a safe and lossless fault-tolerant verification space for the traffic scheduling system. This completely avoids irreversible damage to underlying business data caused by blind physical execution. Simultaneously, this solution creatively introduces the relative entropy (KL divergence) algorithm from information theory, transforming the originally highly abstract and difficult-to-detect concepts of profile distortion and sample skew into a high-frequency, high-precision quantifiable feature offset scalar indicator at the computer's underlying layer. This intelligent control mechanism of "simulation first, verification later, and then release" can accurately circuit-break the risk of severe sample bias caused by exclusive traffic filtering at the millisecond level. It eliminates the bias blind spot of traditional A / B testing that blindly pursues orthogonality at the expense of representativeness, and further ensures that the clean population traffic that ultimately enters the target strategy test always maintains a high degree of isomorphism in its macroscopic feature structure with the initial healthy overall population.

[0087] Optionally, if the feature distribution offset is greater than or equal to the system tolerance threshold, the following methods can also be used during implementation: Step B2: If the feature distribution offset is greater than or equal to the system tolerance threshold, then determine the target feature weight vector that has shifted based on the initial baseline feature distribution and the remaining subject feature distribution; The system tolerance threshold refers to a scalar boundary value (e.g., 0.1 or 0.05) pre-set in the configuration center by the service platform. This value represents the maximum tolerance limit that the experimental sample profile can deviate from the overall system profile due to the removal of overlapping groups, which is acceptable from a business perspective.

[0088] The target feature weight vector refers to a multi-dimensional quantized array or tensor output by the underlying algorithm engine of a computer system through matrix operations. This vector not only accurately locates which dimensions of profile features (such as high net worth, low frequency of activity, etc.) were severely lost during the removal operation, but also quantifies the proportion of each missing feature that needs to be injected in the subsequent compensation operation through normalized weight values.

[0089] Indicatively, the feature distribution offset output from the previous step (S408) is first intercepted, and a strict comparison logic operation is performed between it and a preset system tolerance threshold in memory. If the offset is determined to be greater than or equal to the system tolerance threshold, the feature rebalancing interrupt service is immediately triggered. The underlying feature engineering module is invoked to quickly read the initial baseline feature distribution matrix and the remaining test feature distribution matrix in parallel from the cache cluster. A row-by-row, column-by-column matrix subtraction operator is performed on these two matrices to calculate the probability density difference on each feature dimension. For feature dimensions with a positive difference (i.e., the initial distribution is greater than the remaining distribution, indicating that this type of user has been severely consumed in the virtual culling), the absolute value of the difference is extracted into a candidate feature array; for dimensions with a negative difference or close to zero, zeroing and decay processing is performed. Further, a normalization function (such as L1 norm normalization, Min-Max scaling) is called to transform the candidate feature array into a probability distribution vector with a sum of 1. This vector is then established as the target feature weight vector indicating the offset. The weight vector is serialized and persisted to a distributed message queue, serving as the input parameter for downstream sandbox compensation tasks, thus completing the targeted missing location of the basic profile with extremely low computational overhead.

[0090] Step B4: Based on the target feature weight vector, extract an equal number of clean user entity identifiers that match the feature attributes of the overlapping test user entities from the pre-set global user traffic neutral sandbox. A global user traffic neutral sandbox refers to an environment that is pre-isolated by the service platform in its underlying traffic routing and experimental distribution architecture through hash modulo or whitelist mechanisms (e.g., a fixed 5% of the global traffic pool). This user group is forcibly blocked by the underlying gateway of the system during any regular business cycle and does not participate in any A / B testing or canary release of any business line. This constitutes a clean traffic pool with a feature distribution highly isomorphic to the overall pool and absolutely uncontaminated by any experimental variables.

[0091] Equal number: refers to ensuring that the number of pure users extracted from the sandbox during the sampling compensation process is completely consistent in absolute value with the number of overlapping subject user entities that were virtually removed in step S404.

[0092] Pure User Entity Identifier: refers to the unique underlying data key (such as UID, DeviceID) of the user extracted from the above-mentioned neutral sandbox that meets the profile matching requirements of the target feature weight vector and is in an absolutely orthogonal untested state.

[0093] Step B6: Update the routing distribution configuration of the gateway node and inject the clean user entity identifier compensation into the target test user entity set.

[0094] Compensation injection refers to physically pushing uncontaminated, clean sample identifiers extracted from the sandbox into the ongoing experiment's whitelist by dynamically modifying the memory view or configuring synchronization, thereby achieving an equivalent replacement for the removed overlapping samples.

[0095] Indicatively, the clean user entity identifiers extracted in step B4 are first formatted and encapsulated to generate an incremental compensation list. Then, the scheduling platform initiates a configuration distribution transaction through the configuration center service, online rewriting the definition fields regarding the target test user entity set in the routing distribution configuration of the gateway nodes. Overlapping user identifiers marked as conflicting are completely removed from the whitelist, and clean user entity identifiers are batch-added to the vacant positions in the whitelist. The configuration center uses a long-connection heartbeat mechanism to push the updated configuration snapshot to each distributed gateway node. Upon receiving the change signal, the gateway node triggers the hot-reloading logic of its local configuration, completing the atomic replacement of the in-memory routing mapping table without interrupting existing network connections. This process ensures a smooth transition between bad and good samples from the physical layer, enabling clean users subsequently reaching the gateway to seamlessly inherit the target service strategy.

[0096] The following will combine Figure 3 This specification provides a detailed description of the deployment and testing processing device provided in the embodiments. It should be noted that... Figure 3 The deployment test processing device shown is used to execute the instructions in this manual. Figures 1-2 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 1-2 The example shown.

[0097] Please see Figure 3 This diagram illustrates the structure of a delivery testing processing device according to an embodiment of this specification. The delivery testing processing device 1 can be implemented as all or part of a user terminal through software, hardware, or a combination of both. According to some embodiments, the delivery testing processing device 1 includes a traffic monitoring module 11 and a delivery testing module 12, specifically used for: Traffic monitoring module 11 is used to monitor the service strategy experiment status data of each user traffic delivery path in the service strategy test scenario, and manage the wide table of user traffic delivery experiment records based on the service strategy experiment status data. The delivery test module 12 is used to obtain the target test user entity set selected by the target service policy when receiving the target service policy traffic delivery request to be processed, and to perform experimental user traffic experiment conflict detection on the target test user entity set according to the wide table of user traffic delivery experiment records, so as to remove and isolate overlapping test user entities in the target test user entity set that overlap with the already run test experiment in the experimental conflict state. The delivery testing module 12 is used to periodically monitor the wide table of user traffic delivery experiment records and the set of target test users corresponding to the target service strategy during the operation testing phase after the target service strategy is delivered, and to perform delivery operation experiment conflict detection based on the wide table of user traffic delivery experiment records and the set of target test users, so as to trigger an abnormal warning in the experimental conflict state. The deployment testing module 12 is used to perform experimental data overlap detection on the experimental group and control group of the target service strategy after the experimental test of the target service strategy is completed, obtain service test data overlap information, and generate an experimental data independence test report for the target service strategy based on the service test data overlap information.

[0098] It should be noted that the deployment test processing device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the deployment test processing method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the deployment test processing device and the deployment test processing method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0099] The example numbers in this specification are for descriptive purposes only and do not represent the superiority or inferiority of the examples.

[0100] This specification also provides a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1-2 The deployment test processing method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1-2 The specific details of the illustrated embodiments will not be elaborated here.

[0101] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figures 1-2 The deployment test processing method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1-2 The specific details of the illustrated embodiments will not be elaborated here.

[0102] Please refer to Figure 4 This is a structural block diagram of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 1010, a memory 1020, an input device 1030, an output device 1040, and a bus 1050. The processor 1010, memory 1020, input device 1030, and output device 1040 may be connected to each other via the bus 1050.

[0103] Processor 1010 may include one or more processing cores. Processor 1010 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 1020, and by calling data stored in memory 1020. Optionally, processor 1010 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 1010 may integrate one or more of a central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 1010 and may be implemented separately through a communication chip.

[0104] The memory 1020 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1020 may include non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, code, code sets, or instruction sets.

[0105] The input device 1030 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 1040 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In this embodiment, the input device 1030 can be a temperature sensor for acquiring the operating temperature of the electronic device. The output device 1040 can be a speaker for outputting audio signals.

[0106] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WIFI) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0107] In the embodiments of this specification, the executing entity for each step can be the electronic device described above. Optionally, the executing entity for each step can be the operating system of the electronic device. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0108] exist Figure 4 In the electronic device, the processor 1010 can be used to call a program stored in the memory 1020 and execute it to implement the deployment test processing method as described in the various method embodiments of this specification.

[0109] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0110] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the features, data, and information involved in this specification were all obtained under full authorization.

[0111] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. A method for testing and processing deployment, characterized in that, Applied to a service platform, the method includes: In the service strategy testing scenario, monitor the service strategy experiment status data of each user traffic delivery path, and manage the wide table of user traffic delivery experiment records based on the service strategy experiment status data. When a target service strategy traffic delivery request is received, the target test user entity set selected by the target service strategy is obtained, and the test user traffic experiment conflict detection is performed on the target test user entity set according to the wide table of user traffic delivery experiment records. In the event of an experiment conflict, overlapping test user entities in the target test user entity set that overlap with the already run test experiment are removed and isolated. During the operational testing phase after the target service strategy is deployed, the wide table of user traffic deployment experiment records and the set of target test user entities corresponding to the target service strategy are periodically monitored. Based on the wide table of user traffic deployment experiment records and the set of target test user entities, deployment operation experiment conflict detection is performed to trigger an abnormal warning in the event of an experimental conflict. After the experimental testing of the target service strategy is completed, the experimental data overlap detection is performed on the experimental group and the control group of the target service strategy to obtain service test data overlap information, and an experimental data independence test report for the target service strategy is generated based on the service test data overlap information.

2. The method according to claim 1, characterized in that, The wide table for managing user traffic delivery experiment records based on the service strategy experiment status data includes: The test traffic is parsed on the service strategy experiment status data. A wide table of user traffic delivery experiment records is constructed and updated based on user attribute identifiers, date identifiers, and service experiment test task identifiers. The wide table of user traffic delivery experiment records is used to record all service test path trajectories participated in by each test user entity within the test time period.

3. The method according to claim 1, characterized in that, The step of detecting conflicts in the delivery experiment based on the wide table of user traffic delivery experiment records and the target set of subject users, in order to trigger an anomaly warning in the event of an experimental conflict, includes: Based on the wide table of user traffic delivery experiment records and the target test user entity set, determine the test run experiment overlap information between the target test user entity set and the test entity set of the run experiment; Based on the overlapping information of the test run experiments, the system detects the experimental conflict status during the deployment and operation, so as to trigger an abnormal warning when the experimental conflict status occurs.

4. The method according to claim 3, characterized in that, The detection of experimental conflict states during deployment based on the test run overlap information includes: Based on the test run experiment overlap information, newly added overlapping test experiments are detected, and based on the test run experiment overlap information, the dynamic overlap ratio between the target subject user entity set and the already run experiment subject entity set is determined; If there are newly added overlapping experimental tests and / or the dynamic overlap ratio is greater than the preset safety threshold, it is determined to be an experimental conflict state, and test warning information including conflict entity details is generated and the test warning information is distributed to the test task terminal.

5. The method according to claim 1, characterized in that, The experimental data overlap detection between the experimental group and the control group of the target service strategy is performed to obtain service test data overlap information, including: Calculate the first experimental orthogonal quantification index corresponding to the target experimental group and the second control orthogonal quantification index corresponding to the target control group for the target service strategy; wherein, the first experimental orthogonal quantification index includes the first overlap ratio corresponding to each run test experiment, the first overlap ratio representing the overlap ratio of the subject user entity group corresponding to the run test experiment in the total number of user groups in the target experimental group; the second control orthogonal quantification index includes the second overlap ratio corresponding to each run test experiment, the second overlap ratio representing the overlap ratio of the subject user entity group corresponding to the run test experiment in the total number of user groups in the target control group. Based on the first experimental orthogonal quantification index and the second control orthogonal quantification index, the data distribution independence condition is analyzed to obtain the service test data overlap information.

6. The method according to claim 5, characterized in that, The step of performing data distribution independence condition analysis based on the first experimental orthogonal quantification index and the second control orthogonal quantification index to obtain the service test data overlap information includes: Extract the total number of run test experiments that overlap with the target service strategy; Based on the first experimental orthogonal quantization index and the second control orthogonal quantization index, the data distribution independence deviation value is calculated using the first calculation formula. When the data distribution independence deviation value is determined to be less than the preset independence judgment threshold, service test data overlap information representing that the data distribution independence condition is met is generated. When the data distribution independence deviation value is determined to be greater than or equal to the preset independence judgment threshold, service test data overlap information representing that the data distribution independence condition is not met is generated. The first calculation formula satisfies the following formula: Wherein, Diff represents the data distribution independence deviation value, N represents the total number of test experiments that have been run, Ti represents the first overlap ratio, Ci represents the second overlap ratio, and i is a positive integer and represents the label of the test experiment that has been run.

7. The method according to claim 5, characterized in that, The step of performing data distribution independence condition analysis based on the first experimental orthogonal quantification index and the second control orthogonal quantification index to obtain the service test data overlap information includes: Extract the total number of user entities used in the experimental testing of the target service strategy; Based on the data calculated using the first experimental orthogonal quantization index and the second control orthogonal quantization index, the data distribution independence decay confidence value is calculated using the second calculation formula. When the data distribution independence decay confidence value is less than a preset independence judgment threshold, service test data overlap information representing that the data distribution independence condition is met is generated. When the data distribution independence decay confidence value is greater than or equal to the preset independence judgment threshold, service test data overlap information representing that the data distribution independence condition is not met is generated. The second calculation formula satisfies the following formula: Wherein, Conf is the confidence value for the independence of the data distribution, K represents the total number of user entities in the experimental test, Ti represents the first overlap ratio, Ci represents the second overlap ratio, and i is a positive integer and represents the label of the test experiment that has been run.

8. The method according to claim 1, characterized in that, The step of removing and isolating overlapping test user entities in the target test user entity set that overlap with the already run test experiments includes: Identify overlapping user entities in the target user entity set that overlap with the run test experiments, extract target multidimensional attribute features corresponding to the target user entity set, and determine the initial baseline feature distribution of the target user entity set based on the target multidimensional attribute features; A virtual culling operation is performed on the overlapping subject user entities in memory space, and physical deletion instructions targeting the underlying database are intercepted; Based on the target multidimensional attribute features, the feature distribution of the remaining test user entities after virtually removing the overlapping test user entities is re-statistically analyzed to obtain the remaining test feature distribution; Calculate the relative entropy between the remaining subject feature distribution and the initial baseline feature distribution to obtain the feature distribution offset; Determine whether the feature distribution offset is greater than or equal to a preset system tolerance threshold. If the feature distribution offset is less than the system tolerance threshold, then remove and isolate overlapping test user entities.

9. The method according to claim 8, characterized in that, The method further includes: If the feature distribution offset is greater than or equal to the system tolerance threshold, then the target feature weight vector that has shifted is determined based on the initial baseline feature distribution and the remaining subject feature distribution. Based on the target feature weight vector, an equal number of clean user entity identifiers matching the feature attributes of the overlapping test user entities are extracted from the pre-set global user traffic neutral sandbox. Update the routing distribution configuration of the gateway node and inject the clean user entity identifier compensation into the target subject user entity set.

10. A delivery testing and processing device, characterized in that, The device, applied to a service platform, includes: The traffic monitoring module is used to monitor the service strategy experiment status data of each user traffic delivery path in the service strategy test scenario, and manage the wide table of user traffic delivery experiment records based on the service strategy experiment status data. The delivery testing module is used to obtain the target test user entity set selected by the target service policy when it receives the target service policy traffic delivery request to be processed, and to perform experimental user traffic experiment conflict detection on the target test user entity set according to the wide table of user traffic delivery experiment records, so as to remove and isolate overlapping test user entities in the target test user entity set that overlap with the already run test experiment in the experimental conflict state. The delivery testing module is used to periodically monitor the wide table of user traffic delivery experiment records and the set of target test user entities corresponding to the target service strategy during the operation testing phase after the target service strategy is delivered. Based on the wide table of user traffic delivery experiment records and the set of target test user entities, the module performs delivery operation experiment conflict detection to trigger an abnormal warning in the event of an experimental conflict. The deployment testing module is used to perform experimental data overlap detection on the experimental group and control group of the target service strategy after the experimental test of the target service strategy is completed, obtain service test data overlap information, and generate an experimental data independence test report for the target service strategy based on the service test data overlap information.

11. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions, which are adapted to be loaded by a processor and executed by the steps of the method as described in any one of claims 1 to 9.

12. A computer program product storing at least one instruction, said at least one instruction being loaded by a processor and executing the steps of the method as claimed in any one of claims 1 to 9.

13. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as described in any one of claims 1 to 9.

Images