Pop-up frame ablation rule generation method and device, storage medium and computer equipment

Abstract

According to the pop-up frame ablation rule generation method and device, the storage medium and the computer equipment, when the target application performs interface traversal, whether the pop-up frame exists in the current interface or not is detected, and if yes, similarity evaluation is performed on the current interface and the historical traversal interface. Repeated generation of ablation rules for similar interfaces is avoided, and the generation efficiency is improved. And if the interface set with the similarity with the current interface higher than the preset threshold value does not exist, determining component information corresponding to the pop-up ablation event. In consideration of screen resolution differences of different devices, component information and an interface component tree are subjected to similarity matching; therefore, various devices can be effectively adapted, and the target component can be accurately positioned. And then obtaining a corresponding pop-up ablation rule according to the related information of the target component, and continuing to traverse to finally obtain an ablation rule set of the target application. In this way, the ablation rule can be automatically generated in the testing process through automatic traversal, and the comprehensiveness of pop-up scene coverage can be enhanced while the generation efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device, storage medium and computer equipment for generating a bullet box ablation rule. Background Art

[0002] In the current mobile Internet era, with the vigorous development of various commercial applications, users often encounter various pop-up interfaces when using these applications, such as permission requests, ad pop-ups, update prompts, etc. In order to improve the user experience, many developers have launched custom screen click Android applications, which aim to automatically handle these pop-up interfaces and reduce the user's operation burden.

[0003] However, current custom screen click applications still rely heavily on business personnel to manually collect pop-up interface information of each application and write corresponding pop-up ablation rules based on it. This process is not only time-consuming and laborious, but also difficult to ensure that all possible pop-up situations are fully covered due to the diversity and complexity of pop-up scenarios. Therefore, when generating pop-up ablation rules, there are often dual challenges of accuracy and efficiency. Summary of the invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the custom screen click application in the prior art still relies heavily on business personnel to manually collect the pop-up interface information of each application and write corresponding pop-up ablation rules accordingly. This process is not only time-consuming and laborious, but also due to the diversity and complexity of the pop-up scene, it is difficult to ensure comprehensive coverage of all possible pop-up situations. Therefore, when generating pop-up ablation rules, there are often technical defects that face the dual challenges of accuracy and efficiency.

[0005] In a first aspect, the present application provides a method for generating a bullet box ablation rule, the method comprising:

[0006] When the target application performs interface traversal according to the preset traversal strategy, a pop-up box detection is performed on the current interface. When a pop-up box is detected in the current interface, a similarity evaluation is performed between the current interface and the historical traversal interface;

[0007] If there is no interface set in the historical traversal interfaces whose similarity with the current interface is higher than a preset threshold, then determining component information corresponding to the bullet box ablation event in the bullet box of the current interface;

[0008] Acquire an interface component tree of the current interface, and perform similarity matching between the component information and the interface component tree to determine a target component that causes the bullet box in the current interface to disappear;

[0009] After filling the relevant information of the target component into the preset template to obtain the ablation rule corresponding to the pop-up box, continue to traverse the interface of the target application until all the interfaces to be traversed are traversed, and then generate the ablation rule set of the target application according to the ablation rules obtained during the traversal process.

[0010] In one embodiment, the similarity evaluation between the current interface and the historical traversal interface includes:

[0011] Obtain the root interface of each interface set in the historical traversal interface;

[0012] According to the component texts and component positions in the current interface and each root interface, respectively calculate the similarity between the current interface and each root interface, and obtain the similarity between the current interface and each root interface;

[0013] If the similarity between the current interface and each root interface is not higher than the preset threshold, then there is no interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold; otherwise, there is an interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold.

[0014] In one embodiment, the performing a pop-up window detection on the current interface includes:

[0015] Get the preset pop-up box detection model;

[0016] Input the screenshot of the current interface into the pop-up box detection model. If the output result is empty, it is considered that there is no pop-up box in the current interface. Otherwise, the location information of the pop-up box is obtained.

[0017] According to the position information, a non-pop-up box area is determined in the current interface, and the shadow area ratio in the non-pop-up box area is calculated. If the shadow area ratio is lower than a preset ratio, it is considered that there is no pop-up box in the current interface. Otherwise, it is considered that there is a pop-up box in the current interface.

[0018] In one embodiment, determining the component information corresponding to the bullet box ablation event in the bullet box of the current interface includes:

[0019] Get the position information of the pop-up box to determine the pop-up box area in the current interface;

[0020] Identify clickable components in the pop-up area, and determine user preference options and application preference options in each clickable component based on component text and component contrast;

[0021] The user preference option and the application preference option are used as optimal interaction objects, and components other than the user preference option and the application preference option in each clickable component are used as suboptimal interaction objects;

[0022] Click each clickable component in turn according to the priority until the clickable component that causes the pop-up box to disappear is determined, and the component information of the clickable component is determined as the component information corresponding to the pop-up box disappearance event. When the priorities are the same, the clickable component to be clicked is randomly determined.

[0023] In one embodiment, the obtaining of the interface component tree of the current interface and performing similarity matching between the component information and the interface component tree includes:

[0024] Obtaining an interface component tree of the current interface, and calculating the Euclidean distance between the component information and each component in the interface component tree;

[0025] The component with the smallest Euclidean distance to the component information is determined as the target component for causing the bullet box in the current interface to disappear.

[0026] In one embodiment, the step of filling the relevant information of the target component into a preset template to obtain an ablation rule corresponding to the bullet box includes:

[0027] Acquire the path of the target component in the interface component tree, the text information of the target component, the click attribute of the target component and the interface identifier to which it belongs, and determine the acquired information as the relevant information of the target component;

[0028] A preset template is obtained, and relevant information of the target component is filled into a corresponding position in the preset template to obtain an ablation rule corresponding to the bullet box.

[0029] In one embodiment, the method further comprises:

[0030] Determine the type of pop-up box corresponding to the ablation rule according to the current interface and the component information;

[0031] When determining the pop-up box type of the ablation rule obtained in the traversal process, classifying the ablation rules in the ablation rule set according to the pop-up box type to obtain at least one rule set;

[0032] When a type adjustment instruction from the user is received, a target set is determined in each rule set according to the type adjustment instruction, and a corresponding ablation rule in the target set is turned on or off according to the type adjustment instruction.

[0033] In a second aspect, the present application provides a device for generating a bullet-frame ablation rule, the device comprising:

[0034] The pop-up box detection module is used to detect pop-up boxes on the current interface when the target application traverses the interface according to the preset traversal strategy. When a pop-up box is detected in the current interface, the current interface is evaluated for similarity with the historical traversal interface.

[0035] An information determination module, used to determine component information corresponding to the bullet box ablation event in the bullet box of the current interface if there is no interface set whose similarity with the current interface is higher than a preset threshold in the historical traversal interface;

[0036] A component determination module, used to obtain an interface component tree of the current interface, and perform similarity matching between the component information and the interface component tree to determine a target component that causes the bullet box in the current interface to disappear;

[0037] The rule generation module is used to fill the relevant information of the target component into the preset template to obtain the ablation rule corresponding to the pop-up box, and then continue to traverse the interface of the target application until all the interfaces to be traversed are traversed, and then generate the ablation rule set of the target application according to the ablation rules obtained during the traversal process.

[0038] In a third aspect, the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for generating a bullet-box ablation rule as described in any of the above embodiments.

[0039] In a fourth aspect, the present application provides a computer device, comprising: one or more processors, and a memory;

[0040] The memory stores computer-readable instructions, and when the one or more processors execute the computer-readable instructions, the steps of the method for generating a bullet-box ablation rule as described in any one of the above embodiments are performed.

[0041] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0042] The method, device, storage medium and computer device for generating a bullet-box ablation rules provided by the present application, in the process of generating a bullet-box ablation rules, when the target application performs interface traversal according to the preset traversal strategy, a bullet-box detection is performed on the current interface, and when a bullet-box is detected in the current interface, the current interface is evaluated for similarity with the historical traversal interface. This avoids repeated generation of ablation rules for similar interface bullet-boxes, thereby improving the efficiency of ablation rule generation. When there is no interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold, the component information corresponding to the bullet-box ablation event is determined in the bullet-box of the current interface, and then considering the screen resolution differences of different devices, the interface component tree of the current interface can be obtained at this time, and the component information is matched with the interface component tree for similarity. Since the interface component tree of the same application on different devices is generally stable, it can effectively adapt to different devices and accurately locate the target component that causes the bullet-box ablation. After that, the relevant information of the target component is filled into the preset template to obtain the ablation rule corresponding to the bullet-box, and the interface traversal of the target application is continued until all interfaces to be traversed are traversed to form the ablation rule set of the target application. In this way, through automated traversal, application pop-up box elimination rules can be automatically generated during the test process, which can save manpower and improve generation efficiency while also enhancing the comprehensiveness of pop-up box scenario coverage. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0044] Figure 1 A flowchart of a method for generating a bullet-frame ablation rule provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a process for evaluating the similarity between the current interface and the historical traversal interface provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a process for performing a pop-up window detection on the current interface provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a process for determining component information corresponding to a bullet box ablation event in a bullet box of a current interface provided by an embodiment of the present application;

[0048] Figure 5 An example diagram of a method for generating a bullet-box ablation rule provided in an embodiment of the present application;

[0049] Figure 6 A schematic diagram of the structure of a bullet-frame ablation rule generation device provided in an embodiment of the present application;

[0050] Figure 7 An internal structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] like Figure 1 As shown, the present application provides a method for generating a bullet box ablation rule, the method comprising:

[0053] S101: When the target application performs interface traversal according to a preset traversal strategy, a pop-up window detection is performed on the current interface.

[0054] Among them, the preset traversal strategy refers to a method of using an adaptive depth-first search algorithm to traverse the interface. Specifically, the adaptive depth-first search (ADFS) algorithm can use a depth-first method to traverse the interface. The algorithm optimizes the exploration process by dynamically adjusting the maximum traversal depth. Initially, the maximum traversal depth is set to 1, and a depth-first search (DFS) is performed at this depth. When the proportion of explored clickable components reaches a preset coverage threshold, the algorithm increases the maximum traversal depth (for example, from 1 to 2) to further explore deeper interfaces. If the traversal in a certain direction reaches the current maximum depth, the algorithm will back off and try to explore in other directions. When the traversal in all directions reaches the current maximum depth, the algorithm will increase the maximum traversal depth again and continue to explore in depth until the time budget or coverage requirement is met. This adaptive mechanism ensures that more interface levels are efficiently covered within a limited time. In the scenario of ablation rule generation, using this traversal method can quickly broaden the horizontal interface range generated by the rule. Moreover, since users of shallow interfaces have a higher frequency of use, this method can also give priority to generating ablation rules with a higher frequency of use.

[0055] In addition, when traversing the interface, the interface transition graph is used to ensure that the interface rollback operation proceeds normally. Specifically, when the maximum traversal depth is reached and rollback is required, for the Activity interface, the interface is stored by maintaining a stack, and the rollback operation follows the FILO (first in, last out) rule to return to the previous Activity interface. However, many application interfaces use Fragment interface layouts. Since the Fragment interface is not stored in the stack, a single rollback operation may destroy all previously traversed Fragment interfaces. These Fragment interfaces may still have unclicked components, that is, interfaces that have not been fully traversed. Therefore, to ensure that the rollback operation proceeds normally, the interface transfer graph is dynamically maintained during the interface traversal. The interface transfer graph is used to record the traversed interface path. If the application is restarted or the traversed interface is destroyed during the traversal process, the path can be restored through the interface transfer graph. This ensures that the ablation rule generation proceeds normally.

[0056] Furthermore, for the traversed interface, if there is an interface that has a loop relationship with other interfaces or its corresponding interface sequence is in the same mode for a long time, the interface is marked as a fault interface and avoided during traversal. This can further ensure the smooth progress of the ablation rule generation process.

[0057] In this step, when the user needs to generate a pop-up box ablation rule for the target application, the pop-up box ablation rule can be generated in the process of automated testing of the target application, and the pop-up box ablation rule generation method is executed by using the traversal process of the automated test itself, which not only saves resources but also improves the efficiency of rule generation. Then, in the process of the target application traversing the interface according to the preset traversal strategy, a pop-up box is detected to see if there is a pop-up box in the current interface. Specifically, a pop-up box refers to an overlay that is prominently displayed on the application interface during user interaction, which is used to provide information or request operations to guide user behavior and improve user experience.

[0058] S102: Determine whether there is a pop-up window in the current interface.

[0059] S103: When a pop-up window is detected in the current interface, a similarity evaluation is performed between the current interface and the historical traversal interface.

[0060] The historical traversal interface includes multiple interface sets, and the interface set refers to a set of similar interfaces in the historical traversal. It should be noted that the historical traversal interface refers to the interface containing a pop-up window in the traversal.

[0061] In this step, when a pop-up box is detected in the current interface, the current interface and each interface set in the historical traversal interface can be evaluated for similarity, so that it can be determined whether it is necessary to generate a pop-up box ablation rule for the pop-up box of the current interface by evaluating the interface similarity. It can be understood that the similarity evaluation can evaluate the similarity of two interfaces containing pop-ups.

[0062] Specifically, due to the large number of pages in complex business applications, and the fact that pages related to the same business may only have slight differences, for example, the product interfaces of different e-commerce apps are generally similar. If all interfaces are traversed without screening, there will be a problem of path explosion, and the efficiency of interface traversal will be greatly reduced. Therefore, the similarity between the current interface and the historical traversal interface is calculated. If there is a set of interfaces in the historical traversal interface whose similarity with the current interface is higher than the preset threshold, there is no need to generate ablation rules for the pop-up box of the current interface, so as to save computing resources and improve generation efficiency.

[0063] Furthermore, when evaluating similarity, the root interface of each interface set in the historically traversed interface is used for evaluation, which can avoid evaluating and comparing all historically traversed interfaces one by one, thereby further improving the generation efficiency of ablation rules.

[0064] S104: Determine whether there is an interface set in the historical traversal interfaces whose similarity with the current interface is higher than a preset threshold.

[0065] S105: If there is no interface set in the historical traversal interfaces whose similarity with the current interface is higher than a preset threshold, component information corresponding to the bullet box disappearance event is determined in the bullet box of the current interface.

[0066] The preset threshold is an empirical value and can be adjusted according to actual needs. Similarity is used to measure the similarity between two interfaces.

[0067] When there is no interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold, it means that there is no interface similar to the current interface in the historical traversal interface. Therefore, a pop-up box ablation rule can be generated for the current interface. First, the component information corresponding to the pop-up box ablation event can be determined in the pop-up box of the current interface. Among them, the component information refers to the component information corresponding to the component that can cause the pop-up box to ablate in the pop-up box of the current interface.

[0068] S106: Acquire the interface component tree of the current interface, and perform similarity matching between the component information and the interface component tree to determine a target component that causes the bullet box in the current interface to disappear.

[0069] The interface component tree refers to treating the user interface as a tree structure composed of various components. In this structure, each component is a node on the tree, and the relationship between components forms the tree structure.

[0070] In this step, since the resolutions of different devices generally vary to a certain extent, the coordinate information in the obtained component information will also be inconsistent. Using only the component information corresponding to the component that can cause the pop-up box to disappear may not necessarily accurately locate the target component. The interface component tree of the same application in different devices is generally stable. Therefore, in order to enable the generated ablation rules to be reused in multiple devices, the interface component tree of the current interface can be obtained, and the component information can be matched with the interface component tree for similarity to determine the node closest to the component information in the interface component tree, and then the component corresponding to the interface can be determined as the target component that causes the pop-up box in the current interface to disappear.

[0071] S107: After filling the relevant information of the target component into the preset template to obtain the ablation rule corresponding to the pop-up box, continue to traverse the interface of the target application until all the interfaces to be traversed are traversed, and then generate an ablation rule set for the target application according to the ablation rules obtained during the traversal process.

[0072] The relevant information includes, but is not limited to, the path information of the target component on the interface component tree, the interface identifier to which the target component belongs, and the like.

[0073] In this step, when the target component is determined, a bullet box ablation rule can be generated for the current interface bullet box. By obtaining the relevant information of the target component and filling the relevant information into the corresponding position in the preset template, the ablation rule corresponding to the bullet box of the current interface can be obtained. Then, the interface of the target application is traversed until all the interfaces to be traversed determined according to the preset traversal strategy are traversed, and the ablation rule set of the target application is generated according to the ablation rules obtained during the traversal process.

[0074] It can be understood that in the ablation rule application stage, the ablation rule set of the target application can be imported into the preset ablation rule matching module. When the user uses the target application, if the interface changes, the current interface component tree will be obtained and then matched with the ablation rules in the ablation rule set. When a certain ablation rule is matched, the ablation rule is executed to ablate the pop-up box of the current interface.

[0075] S108: If there is no pop-up box in the current interface or there is an interface set in the historical traversal interface whose similarity with the current interface is higher than a preset threshold, continue to traverse the interface of the target application until all the interfaces to be traversed are traversed, and then generate a set of ablation rules for the target application according to the ablation rules obtained during the traversal process.

[0076] The method, device, storage medium and computer device for generating a bullet-box ablation rules provided by the present application, in the process of generating a bullet-box ablation rules, when the target application performs interface traversal according to the preset traversal strategy, a bullet-box detection is performed on the current interface, and when a bullet-box is detected in the current interface, the current interface is evaluated for similarity with the historical traversal interface. This avoids repeated generation of ablation rules for similar interface bullet-boxes, thereby improving the efficiency of ablation rule generation. When there is no interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold, the component information corresponding to the bullet-box ablation event is determined in the bullet-box of the current interface, and then considering the screen resolution differences of different devices, the interface component tree of the current interface can be obtained at this time, and the component information is matched with the interface component tree for similarity. Since the interface component tree of the same application on different devices is generally stable, it can effectively adapt to different devices and accurately locate the target component that causes the bullet-box ablation. After that, the relevant information of the target component is filled into the preset template to obtain the ablation rule corresponding to the bullet-box, and the interface traversal of the target application is continued until all interfaces to be traversed are traversed to form the ablation rule set of the target application. In this way, through automated traversal, application pop-up box elimination rules can be automatically generated during the test process, which can save manpower and improve generation efficiency while also enhancing the comprehensiveness of pop-up box scenario coverage.

[0077] like Figure 2 As shown, in one embodiment, the current interface and the historical traversal interface are evaluated for similarity, including:

[0078] S201: Obtain the root interface of each interface set in the historical traversal interface.

[0079] Each interface set corresponds to a root interface, and the root interface refers to an interface corresponding to an interface framework common to each interface in the corresponding interface set.

[0080] S202: Calculate the similarity between the current interface and each root interface according to the component text and component position in the current interface and each root interface, and obtain the similarity between the current interface and each root interface.

[0081] Among them, component text refers to the text content displayed by the component, and component position refers to the layout and positioning of the component on the interface.

[0082] In this step, the longest common subsequence algorithm can be used to calculate the similarity between the current interface and each root interface in the component text and component position dimensions respectively, and then the similarity between the current interface and each root interface is obtained by weighted calculation of the similarities of different components in the component text and component position dimensions.

[0083] S203: Determine whether there is a root interface among the root interfaces whose similarity to the current interface is higher than a preset threshold.

[0084] S204: If the similarity between the current interface and each root interface is not higher than the preset threshold, then there is no interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold.

[0085] S205: If there is a root interface in each root interface whose similarity with the current interface is higher than a preset threshold, then there is an interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold.

[0086] Specifically, when the similarity between the current interface and each root interface is not higher than the preset threshold, it means that there is no similar interface to the current interface in the historical traversal interface, that is, there is no set of interfaces in the historical traversal interface whose similarity with the current interface is higher than the preset threshold. When there is one or more root interfaces with a similarity higher than the preset threshold to the current interface, it means that there is at least one similar interface to the current interface in the historical traversal interface, that is, there is a set of interfaces in the historical traversal interface whose similarity with the current interface is higher than the preset threshold.

[0087] In this embodiment, since similar interfaces have been pre-divided into an interface set in the historical traversal interface, the root interface can be used to represent the interface set and perform similarity evaluation with the current interface during the similarity evaluation. This can avoid the need to evaluate all interfaces in the historical traversal interface one by one during the similarity evaluation, thereby avoiding repeated evaluation of similar interfaces, saving computing resources, and improving the efficiency of ablation rule generation.

[0088] like Figure 3 As shown, in one embodiment, performing a pop-up window detection on the current interface includes:

[0089] S301: Obtain a preset pop-up box detection model.

[0090] Among them, the pop-up box detection model is used to detect whether there is a pop-up box in the interface. When there is a pop-up box in the interface, the position information of the pop-up box in the interface is output, and when there is no pop-up box in the interface, the output is empty.

[0091] In one example, the YOLO model can be used as a pre-trained model, and the pre-trained model is iteratively trained using the interface sample set and the bullet box annotation information corresponding to each interface sample in the interface sample set, and the final model is determined as the bullet box detection model. It can be understood that due to the high performance advantage of the YOLO model, the bullet box detection model has higher detection performance and detection efficiency.

[0092] S302: Input the screenshot of the current interface into the pop-up box detection model to determine whether the output result is empty.

[0093] S303: If the output result is empty, it is considered that there is no pop-up box in the current interface.

[0094] S304: If the output result is not empty, obtain the position information of the pop-up window.

[0095] When the output result is not empty, it means that the bullet box detection model has detected a bullet box in the current interface, and the location information of the bullet box output by the bullet box detection model is obtained. Then, in order to avoid false positives of the bullet box detection, further detection and judgment can be performed, that is, the subsequent process of S305-S308.

[0096] S305: Determine a non-pop-up box area in the current interface according to the position information, and calculate a shadow area ratio in the non-pop-up box area.

[0097] The non-bullet box area refers to the remaining area of ​​the interface excluding the area where the bullet box is located. The shadow area ratio refers to the ratio of the shadow area to the non-bullet box area.

[0098] In this step, since the interface with a pop-up box is generally covered with a semi-transparent gray mask in the non-pop-up box area to highlight the pop-up box content, when there is no pop-up box in the interface, but the pop-up box detection model outputs the location information of the pop-up box, the shadow area ratio of the non-pop-up box area should be 0. When there is a pop-up box in the interface, but the location information of the pop-up box output by the pop-up box detection model is greatly different from the actual location of the pop-up box, the shadow area ratio of the non-pop-up box area should be at a relatively small value relative to 100%. Therefore, the accuracy of the pop-up box detection result can be further checked by judging whether the shadow area ratio of the non-pop-up box area is higher than the preset ratio.

[0099] S306: Determine whether the shadow area ratio is lower than a preset ratio.

[0100] Among them, the preset ratio is an empirical value and can be adjusted according to actual conditions.

[0101] S307: If the shadow area ratio is lower than the preset ratio, it is considered that there is no pop-up box in the current interface or the position information error of the detected pop-up box is too large.

[0102] S308: If the shadow area ratio is not less than a preset ratio, it is considered that a pop-up box exists in the current interface.

[0103] In this embodiment, on the one hand, a higher-performance pop-up box detection model is used to achieve the purpose of quickly detecting pop-up boxes. On the other hand, a shadow area ratio judgment strategy is set to further ensure the reliability of the output results of the pop-up box detection model, reduce the possibility of false alarms, and thus improve the accuracy of pop-up box recognition.

[0104] like Figure 4As shown, in one embodiment, determining component information corresponding to the bullet box ablation event in the bullet box of the current interface includes:

[0105] S401: Acquire the position information of the pop-up box to determine the pop-up box area in the current interface.

[0106] The bullet box area refers to the area occupied by the bullet box in the interface.

[0107] S402: Identify clickable components in the pop-up area, and determine user preference options and application preference options in each clickable component based on component text and component contrast.

[0108] Among them, the user preference option refers to the option component that the user wants to select, and the application preference option refers to the option component that the application wants the user to select. For example, the user preference option includes but is not limited to a cross mark, skip, cancel and other buttons, and the application preference option includes but is not limited to confirm, receive and other buttons. In general, the application preference option has a higher contrast than the user preference option.

[0109] In this step, by identifying the clickable components in the pop-up area, and since the user preference options and the application preference options generally have large differences in component text and component contrast, the user preference options and the application preference options can be determined in each clickable component in combination with the component text and component contrast corresponding to each clickable component.

[0110] Specifically, the above process can also be achieved by training a deep learning model. When a third-party framework is used to render a pop-up box, the information contained in the interface component tree will be relatively messy. If the clickable components in the pop-up box area are determined directly through the interface component tree, it may cause the final clickable components to include clickable components in non-pop-up box areas. Therefore, the clickable components in the pop-up box area can be identified through a trained deep learning model to solve the above problem. In one example, as a preference, the YOLO model can be used as a pre-trained model for training a deep learning model.

[0111] S403: The user preference option and the application preference option are taken as optimal interaction objects, and the components other than the user preference option and the application preference option in each clickable component are taken as suboptimal interaction objects.

[0112] It can be understood that the priority of the optimal interaction object is higher than the priority of the suboptimal interaction object.

[0113] In this step, since the user preference option and the application preference option are the components most likely to close the pop-up window, their priorities are set higher than those of other components.

[0114] S404: Click each clickable component in order according to the priority until the clickable component that causes the bullet box to disappear is determined, and then the component information of the clickable component is determined as the component information corresponding to the bullet box disappearance event.

[0115] Among them, when the priorities are the same, the clickable component to be clicked is randomly determined.

[0116] In this embodiment, by setting the priority, the component information that can correspond to the bullet box ablation event can be determined as quickly as possible, thereby improving the generation efficiency of the ablation rule.

[0117] In one embodiment, obtaining an interface component tree of the current interface and performing similarity matching between component information and the interface component tree include:

[0118] S1: Obtain the interface component tree of the current interface, and calculate the Euclidean distance between the component information and each component in the interface component tree.

[0119] Among them, the Euclidean distance is used to measure the matching degree between component information and nodes in the interface component tree.

[0120] S2: Determine the component with the smallest Euclidean distance to the component information as the target component for eliminating the bullet box in the current interface.

[0121] In this embodiment, the component information is matched with each node in the interface component tree to determine the node with the highest matching degree, and the component corresponding to the node is determined as the target component. This can accommodate the position differences between different devices due to resolution differences, so that the generated ablation rules can be applied in multiple devices to improve the reuse rate of the ablation rules.

[0122] In one embodiment, relevant information of the target component is filled into a preset template to obtain an ablation rule corresponding to the bullet box, including:

[0123] S1: Obtain the path of the target component in the interface component tree, the text information of the target component, the click attribute of the target component and the interface identifier to which it belongs, and determine the obtained information as the relevant information of the target component.

[0124] The click attribute of the target component is used to identify whether the component is clickable, and the belonging interface identifier of the target component refers to the identifier of the Activity corresponding to the component. Activity is a component with a user interface, which is used to implement specific user interaction functions.

[0125] S2: Obtain a preset template, and fill relevant information of the target component into a corresponding position in the preset template to obtain an ablation rule corresponding to the pop-up box.

[0126] In this implementation, various information of the target component is used as relevant information of the target component, which can improve the granularity of the matching and thus enhance the matching accuracy in the ablation rule application stage.

[0127] In one embodiment, the method for generating a bullet box ablation rule further includes:

[0128] S1: Determine the type of pop-up box corresponding to the ablation rule according to the current interface and component information.

[0129] Among them, one pop-up box corresponds to one ablation rule, that is, one ablation rule corresponds to one pop-up box type. The pop-up box type refers to the type classified according to the purpose of the pop-up box. The pop-up box type includes but is not limited to the advertising pop-up box, permission pop-up box, prompt pop-up box, etc.

[0130] In this step, the purpose of the pop-up box is determined by analyzing the current interface and component information, thereby determining the pop-up box type corresponding to the ablation rule of the pop-up box.

[0131] S2: When determining the pop-up box type of the ablation rule obtained in the traversal process, classify the ablation rules in the ablation rule set according to the pop-up box type to obtain at least one rule set.

[0132] S3: When receiving a type adjustment instruction from the user, determining a target set in each rule set according to the type adjustment instruction, and turning on or off a corresponding ablation rule in the target set according to the type adjustment instruction.

[0133] The target set refers to a rule set corresponding to the type of pop-up box that the user wants to adjust whether to enable.

[0134] In this embodiment, the ablation rules in the ablation rule set are classified according to the bullet box type, so that the user can turn on or off the application of ablation rules of different bullet box types according to personal wishes, so as to improve the flexibility of ablation rule application.

[0135] like Figure 5 As shown, Figure 5 This is an example diagram of a method for generating a bullet box ablation rule provided in an embodiment of the present application. Figure 5In the method, the execution module of the pop-up box ablation rule generation method can be divided into a pop-up box recognition module, a pop-up box ablation detection module, a pop-up box ablation rule generation module and a dynamic test agent, wherein the target detection module in the pop-up box recognition module is used to identify the pop-up box area in the interface, opacity analysis refers to judging the proportion of the shadow area, the pop-up box ablation detection module is used to identify the clickable components in the pop-up box area, and determine the user preference options and application preference options in the pop-up box area, the pop-up box classification module in the pop-up box ablation rule generation module is used to classify the ablation rules according to the pop-up box type, the UI element matching module is used to match and determine the target component, the state abstraction strategy in the dynamic test agent refers to dividing the historical traversal interface into multiple interface sets to avoid repeated generation of ablation rules, and the fault-tolerant strategy refers to maintaining the interface transition diagram and other means to ensure the smooth progress of rule generation. In addition, the pop-up box context includes the component information corresponding to the pop-up box and the interface component tree.

[0136] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0137] The following describes a bullet-box ablation rule generation device provided in an embodiment of the present application. The bullet-box ablation rule generation device described below and the bullet-box ablation rule generation method described above can refer to each other.

[0138] like Figure 6 As shown, the present application provides a bullet-frame ablation rule generation device 500, the device comprising:

[0139] The pop-up box detection module 501 is used to perform pop-up box detection on the current interface when the target application performs interface traversal according to the preset traversal strategy. When a pop-up box is detected in the current interface, the current interface is evaluated for similarity with the historical traversal interface.

[0140] The information determination module 502 is used to determine the component information corresponding to the bullet box ablation event in the bullet box of the current interface if there is no interface set whose similarity with the current interface is higher than a preset threshold in the historical traversal interface;

[0141] The component determination module 503 is used to obtain the interface component tree of the current interface, and perform similarity matching between the component information and the interface component tree to determine the target component that causes the bullet box in the current interface to disappear;

[0142] The rule generation module 504 is used to fill the relevant information of the target component into the preset template to obtain the ablation rule corresponding to the pop-up box, and then continue to traverse the interface of the target application until all the interfaces to be traversed are traversed, and then generate the ablation rule set of the target application according to the ablation rules obtained during the traversal process.

[0143] In one embodiment, the pop-up box detection module includes:

[0144] The interface acquisition submodule is used to obtain the root interface of each interface set in the historical traversal interface;

[0145] The similarity calculation submodule is used to calculate the similarity between the current interface and each root interface according to the component text and component position in the current interface and each root interface, and obtain the similarity between the current interface and each root interface;

[0146] The judgment submodule is used to determine if the similarity between the current interface and each root interface is not higher than a preset threshold, then there is no interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold; otherwise, there is an interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold.

[0147] In one embodiment, the pop-up box detection module includes:

[0148] The model acquisition submodule is used to obtain the preset pop-up box detection model;

[0149] The pop-up box detection submodule is used to input the screenshot of the current interface into the pop-up box detection model. If the output result is empty, it is considered that there is no pop-up box in the current interface. Otherwise, the location information of the pop-up box is obtained;

[0150] The pop-up box check submodule is used to determine the non-pop-up box area in the current interface according to the position information, and calculate the shadow area ratio in the non-pop-up box area. If the shadow area ratio is lower than the preset ratio, it is considered that there is no pop-up box in the current interface. Otherwise, it is considered that there is a pop-up box in the current interface.

[0151] In one embodiment, the information determination module includes:

[0152] The area determination submodule is used to obtain the position information of the bullet box to determine the bullet box area in the current interface;

[0153] A component identification submodule is used to identify clickable components in the pop-up area, and determine user preference options and application preference options in each clickable component based on component text and component contrast;

[0154] A priority determination submodule, used to take the user preference option and the application preference option as optimal interaction objects, and take the components other than the user preference option and the application preference option in each clickable component as suboptimal interaction objects;

[0155] The component click submodule is used to click on each clickable component in sequence according to the priority until the clickable component that causes the pop-up box to disappear is determined, and the component information of the clickable component is determined as the component information corresponding to the pop-up box disappearance event. When the priorities are the same, the clickable component to be clicked is randomly determined.

[0156] In one embodiment, the component determination module includes:

[0157] The distance calculation submodule is used to obtain the interface component tree of the current interface and calculate the Euclidean distance between the component information and each component in the interface component tree;

[0158] The component determination submodule is used to determine the component with the smallest Euclidean distance to the component information as the target component for eliminating the bullet box in the current interface.

[0159] In one embodiment, the rule generation module includes:

[0160] The information acquisition submodule is used to acquire the path of the target component in the interface component tree, the text information of the target component, the click attribute of the target component and the interface identifier to which it belongs, and determine the acquired information as the relevant information of the target component;

[0161] The rule generation submodule is used to obtain a preset template and fill the relevant information of the target component into the corresponding position in the preset template to obtain the ablation rule corresponding to the bullet box.

[0162] In one embodiment, the bullet frame ablation rule generating device further includes:

[0163] A type determination module is used to determine the type of pop-up box corresponding to the ablation rule according to the current interface and component information;

[0164] A rule classification module, used for classifying the ablation rules in the ablation rule set according to the type of the pop-up box when determining the pop-up box type of the ablation rule obtained in the traversal process, to obtain at least one rule set;

[0165] The rule switch module is used to determine the target set in each rule set according to the type adjustment instruction when receiving the user's type adjustment instruction, and to turn on or off the corresponding ablation rule in the target set according to the type adjustment instruction.

[0166] The division of each module in the above-mentioned bullet-box ablation rule generation device is only for illustration. In other embodiments, the bullet-box ablation rule generation device can be divided into different modules as needed to complete all or part of the functions of the above-mentioned bullet-box ablation rule generation device. Each module in the above-mentioned bullet-box ablation rule generation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0167] In one embodiment, the present application also provides a storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the bullet-box ablation rule generation method as described in any of the above embodiments.

[0168] In one embodiment, the present application also provides a computer device, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the bullet-box ablation rule generation method as described in any of the above embodiments.

[0169] Indicatively, Figure 7 As shown, Figure 7 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. The computer device 600 may be provided as a server. Figure 7 The computer device 600 includes a processing component 602, which further includes one or more processors, and a memory resource represented by a memory 601, for storing instructions executable by the processing component 602, such as an application. The application stored in the memory 601 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 602 is configured to execute instructions to perform the method for generating a bullet box ablation rule in any of the above embodiments.

[0170] The computer device 600 may further include a power supply component 603 configured to perform power management of the computer device 600, a wired or wireless network interface 604 configured to connect the computer device 600 to a network, and an input / output (I / O) interface 605. The computer device 600 may operate based on an operating system stored in the memory 601, such as Windows Server TM, Mac OS X TM, Unix TM, Linux TM, Free BSD TM, or the like.

[0171] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0172] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish an entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only include those elements, but also include other elements that are not clearly listed, or also include elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements limited by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements. Herein, the singular "one", "one" and "described / the" may also include plural forms, unless the context clearly indicates another way. It should also be understood that the terms "include / comprise" or "have" etc. specify the existence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the existence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof. At the same time, the term "and / or" used in this specification includes any and all combinations of the relevant listed items.

[0173] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can refer to each other.

[0174] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a bullet box ablation rule, characterized in that: The method comprises: When the target application performs interface traversal according to the preset traversal strategy, a pop-up box detection is performed on the current interface. When a pop-up box is detected in the current interface, a similarity evaluation is performed between the current interface and the historical traversal interface; If there is no interface set in the historical traversal interfaces whose similarity with the current interface is higher than a preset threshold, then determining component information corresponding to the bullet box ablation event in the bullet box of the current interface; Acquire an interface component tree of the current interface, and perform similarity matching between the component information and the interface component tree to determine a target component that causes the bullet box in the current interface to disappear; After filling the relevant information of the target component into the preset template to obtain the ablation rule corresponding to the pop-up box, continue to traverse the interface of the target application until all the interfaces to be traversed are traversed, and then generate the ablation rule set of the target application according to the ablation rules obtained during the traversal process.

2. The method for generating bullet-frame ablation rules according to claim 1, characterized in that: The similarity evaluation between the current interface and the historical traversal interface includes: Obtain the root interface of each interface set in the historical traversal interface; According to the component texts and component positions in the current interface and each root interface, respectively calculate the similarity between the current interface and each root interface, and obtain the similarity between the current interface and each root interface; If the similarity between the current interface and each root interface is not higher than the preset threshold, then there is no interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold; otherwise, there is an interface set in the historical traversal interface whose similarity with the current interface is higher than the preset threshold.

3. The method for generating bullet-frame ablation rules according to claim 1, characterized in that: The pop-up window detection on the current interface includes: Get the preset pop-up box detection model; Input the screenshot of the current interface into the pop-up box detection model. If the output result is empty, it is considered that there is no pop-up box in the current interface. Otherwise, the location information of the pop-up box is obtained. According to the position information, a non-pop-up box area is determined in the current interface, and the shadow area ratio in the non-pop-up box area is calculated. If the shadow area ratio is lower than a preset ratio, it is considered that there is no pop-up box in the current interface. Otherwise, it is considered that there is a pop-up box in the current interface.

4. The method for generating bullet-frame ablation rules according to claim 1, characterized in that: The determining, in the bullet box of the current interface, component information corresponding to the bullet box disappearance event includes: Get the position information of the pop-up box to determine the pop-up box area in the current interface; Identify clickable components in the pop-up area, and determine user preference options and application preference options in each clickable component based on component text and component contrast; The user preference option and the application preference option are used as optimal interaction objects, and components other than the user preference option and the application preference option in each clickable component are used as suboptimal interaction objects; Click each clickable component in turn according to the priority until the clickable component that causes the pop-up box to disappear is determined, and the component information of the clickable component is determined as the component information corresponding to the pop-up box disappearance event. When the priorities are the same, the clickable component to be clicked is randomly determined.

5. The method for generating bullet-frame ablation rules according to claim 1, characterized in that: The obtaining of the interface component tree of the current interface and performing similarity matching between the component information and the interface component tree includes: Obtaining an interface component tree of the current interface, and calculating the Euclidean distance between the component information and each component in the interface component tree; The component with the smallest Euclidean distance to the component information is determined as the target component for eliminating the bullet box in the current interface.

6. The method for generating bullet-frame ablation rules according to claim 1, characterized in that: Filling the relevant information of the target component into a preset template to obtain an ablation rule corresponding to the bullet box includes: Acquire the path of the target component in the interface component tree, the text information of the target component, the click attribute of the target component and the interface identifier to which it belongs, and determine the acquired information as the relevant information of the target component; A preset template is obtained, and relevant information of the target component is filled into a corresponding position in the preset template to obtain an ablation rule corresponding to the bullet box.

7. The method for generating a bullet-frame ablation rule according to any one of claims 1 to 6, characterized in that: The method further comprises: Determine the type of pop-up box corresponding to the ablation rule according to the current interface and the component information; When determining the pop-up box type of the ablation rule obtained in the traversal process, classifying the ablation rules in the ablation rule set according to the pop-up box type to obtain at least one rule set; When a type adjustment instruction from the user is received, a target set is determined in each rule set according to the type adjustment instruction, and a corresponding ablation rule in the target set is turned on or off according to the type adjustment instruction.

8. A bullet-frame ablation rule generation device, characterized in that: The device comprises: The pop-up box detection module is used to detect pop-up boxes on the current interface when the target application traverses the interface according to the preset traversal strategy. When a pop-up box is detected in the current interface, the current interface is evaluated for similarity with the historical traversal interface. An information determination module, used to determine component information corresponding to the bullet box ablation event in the bullet box of the current interface if there is no interface set whose similarity with the current interface is higher than a preset threshold in the historical traversal interface; A component determination module, used to obtain an interface component tree of the current interface, and perform similarity matching between the component information and the interface component tree to determine a target component that causes the bullet box in the current interface to disappear; The rule generation module is used to fill the relevant information of the target component into the preset template to obtain the ablation rule corresponding to the pop-up box, and then continue to traverse the interface of the target application until all the interfaces to be traversed are traversed, and then generate the ablation rule set of the target application according to the ablation rules obtained during the traversal process.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for generating a bullet-box ablation rule as described in any one of claims 1 to 7.

10. A computer device, characterized in that: include: one or more processors, and memory; The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the method for generating a bullet-box ablation rule according to any one of claims 1 to 7 are performed.

Images