Method and system for verifying mobile application business processes

Abstract

The application provides a mobile application business process verification method and system, and relates to the field of software automation testing. In order to solve the problem that in the related art, the verification use case is batch-mismatched due to unstable control identification across versions, and the business failure caused by the front-end jump due to the server rejection is misreported as passing, the method obtains an intent sequence of a business process to be verified, inputs a control tree of a current interface and an interface screenshot into a page semantic alignment model, obtains a target control matched with the current intent and independent of control identification and coordinates, and performs an operation; within a time window of the operation, interface state changes, server responses and data state fields are jointly collected in the granularity of the intent, the intent is determined to pass when both the interface satisfaction result and the contract satisfaction result meet, otherwise, the intent is determined to not pass and a contract misalignment type is output. The application enables the verification use case to be reused without rewriting for each version, and the business failure caused by the front-end jump due to the server rejection is intercepted and attributed.

Description

Technical Field

[0001] This application relates to the field of software automated testing, and in particular to a method and system for verifying mobile application business processes. Background Technology

[0002] When performing functional regression testing on the business processes of mobile applications, the relevant technologies typically bind each operation step in the process to the identifier of a control on the page to locate the operation target. This identifier can be the resource identifier of the control, the coordinates of the control on the screen, or the text carried by the control. After the operation is completed, the relevant technologies determine whether the step has passed based on whether the expected text or image appears in the visible content on the page.

[0003] This approach is based on the premise that the same operation step corresponds to the same stable control identifier in different versions of the application, and whether a step is successful can be fully reflected by the visible content of the front-end page.

[0004] In multi-step business processes where there are prerequisite state dependencies between steps and the server participates in business decision-making, the above premise no longer holds. Firstly, the resource identifiers, coordinates, and text of controls can change under different builds, experimental distributions, languages, and device models for the same application. Steps bound to specific control identifiers can no longer locate their original operation targets after the application interface is redesigned. Secondly, in this type of business process, whether a step is truly completed is partly determined by the server's response or data status, rather than the visible content of the front-end page.

[0005] The resulting contradiction is this: when the application frontend has displayed a successful submission and completed the page redirection, but the server actually rejects the submission due to validation failure, the frontend's judgment of visible content will still classify this step as successful. This step actually fails in terms of business logic, but the validation report shows it as successful.

[0006] Ultimately, the cross-version verification scripts became incompatible in batches after the interface was redesigned, requiring rewriting for each version. At the same time, steps that were rejected by the server but redirected by the front end were continuously falsely reported as passed, causing regression verification to fail to intercept business operations that should have been blocked, and the verification conclusions deviated from the actual business results. Summary of the Invention

[0007] This application provides a method and system for verifying mobile application business processes, in order to overcome the problems in related technologies where verification test cases suffer from batch mismatches due to the instability of control identifiers across versions, and where the business process that has already been redirected by the front end is mistakenly reported as passed even though the server has rejected it.

[0008] According to a first aspect of the embodiments of this application, a method for verifying a mobile application business process is provided, the method comprising: Obtain the intent sequence of the business process to be verified. Each intent in the intent sequence is associated with a corresponding evidence set, which includes interface evidence and server-side contract evidence. For the intent to be executed, the control tree and screenshot of the current interface are collected. The control tree and the screenshot are input into the page semantic alignment model to obtain the target control that matches the intent to be executed. The determination of the target control does not depend on the resource identifier and coordinates of the control. Execute the operation corresponding to the currently pending intent on the target control; Within the time window of the operation, the first evidence and the second evidence are jointly collected at the granularity of the current intent to be executed. The first evidence is the state change of the current interface before and after the operation is executed, and the second evidence is the server response field triggered by the operation and the server data status field associated with the operation. The first piece of evidence is compared with the interface evidence of the current intent to be executed to obtain an interface satisfaction result; the second piece of evidence is compared with the server-side contract evidence of the current intent to be executed to obtain a contract satisfaction result. When both the interface satisfaction result and the contract satisfaction result are satisfied, the current intent to be executed is determined to be successful; when either the interface satisfaction result or the contract satisfaction result is unsatisfied, the current intent to be executed is determined to be unsatisfied, and based on the combination of the interface satisfaction result and the contract satisfaction result, the contract misalignment type corresponding to the current intent to be executed is output.

[0009] In one exemplary embodiment of this application, after obtaining the intent sequence of the business process to be verified, the method further includes: representing the business process to be verified as a directed step graph with preconditions, wherein each node in the directed step graph corresponds to an intent, and the node is marked with an entry gate required to enter the node, wherein the entry gate is the server-side data state that the preceding node should achieve; before collecting the control tree and screenshot of the current interface, verifying whether the entry gate of the node corresponding to the intent to be executed is achieved; when the entry gate is not achieved, determining the missing preceding node based on the directed step graph, and executing the intent corresponding to the preceding node until the entry gate is achieved.

[0010] In one exemplary embodiment of this application, obtaining the target control that matches the currently to-be-executed intent includes: normalizing the candidate controls in the control tree into business semantic fingerprints, wherein the business semantic fingerprints include the role, reachability label, and value range of the candidate controls; comparing the currently to-be-executed intent with the business semantic fingerprints, and using the candidate controls whose business semantic fingerprints match the intent as the target controls.

[0011] In one exemplary embodiment of this application, after normalizing the candidate controls into business semantic fingerprints, the method further includes: before executing the business process to be verified, establishing a semantic index for the current interface and caching the correspondence between the intent and the target control as an intent control fingerprint; when executing the business process to be verified, matching the controls of the current interface with the intent control fingerprint; when the matching is successful, using the matched control as the target control; when the matching fails, inputting the control tree and the interface screenshot into the page semantic alignment model to obtain the target control, and updating the intent control fingerprint with the target control.

[0012] In an exemplary embodiment of this application, the step of outputting the contract misalignment type corresponding to the currently pending intent includes: when the interface satisfies the result and the contract satisfies the result, the contract misalignment type is determined to be that the front-end performance is passed and the server rejects it; when the interface satisfies the result and the contract satisfies the result, the contract misalignment type is determined to be that the server has processed it and the front-end has not reflected it; based on the fields that cause the non-satisfaction in the first evidence and the second evidence, the intent subsequence that causes the currently pending intent to fail and the fields are extracted as a reproduction slice output.

[0013] In one exemplary embodiment of this application, the method further includes: when the current interface or the server-side contract evidence changes, determining the set of intents affected by the change based on the correspondence between the intent and the business semantic fingerprint; updating the target control corresponding to the intents that fail to match in the intent set, while keeping the target controls corresponding to intents outside the intent set unchanged.

[0014] According to a second aspect of the embodiments of this application, a verification system for mobile application business processes is provided, the system comprising: The intent acquisition module is used to acquire the intent sequence of the business process to be verified. Each intent in the intent sequence is associated with a corresponding evidence set, which includes interface evidence and server-side contract evidence. The intent localization module is used to collect the control tree and screenshot of the current interface for the intent to be executed, input the control tree and the screenshot into the page semantic alignment model, and obtain the target control that matches the intent to be executed. The determination of the target control does not depend on the resource identifier and coordinates of the control. An execution module is used to perform the operation corresponding to the currently pending intent on the target control; The evidence collection module is used to jointly collect first evidence and second evidence within the time window of the operation, with the current intention to be executed as the granularity. The first evidence is the state change of the current interface before and after the operation is executed, and the second evidence is the server response field triggered by the operation and the server data status field associated with the operation. The determination module is used to compare the first evidence with the interface evidence to obtain an interface satisfaction result, compare the second evidence with the server contract evidence to obtain a contract satisfaction result, determine that the current intent to be executed is passed when both the interface satisfaction result and the contract satisfaction result are satisfied, determine that the current intent to be executed is not passed when there is a non-satisfaction, and output the contract misalignment type corresponding to the current intent to be executed based on the combination of the interface satisfaction result and the contract satisfaction result.

[0015] In one exemplary embodiment of this application, the system further includes a gating module; the gating module is used to represent the business process to be verified as a directed step graph with preconditions, each node in the directed step graph corresponding to an intent, the node being marked with an entry gate required to enter the node, the entry gate being the server-side data state that the preceding node should achieve; the gating module is also used to verify whether the entry gate of the node corresponding to the intent to be executed is achieved before the intent location module collects the control tree and screenshot of the current interface, and if the entry gate is not achieved, to determine the missing preceding node according to the directed step graph, and to trigger the execution module to execute the intent corresponding to the preceding node until the entry gate is achieved.

[0016] In an exemplary embodiment of this application, the intent localization module is further configured to: normalize the candidate controls in the control tree into business semantic fingerprints, the business semantic fingerprints including the role, reachability label, and value range of the candidate controls; before executing the business process to be verified, cache the correspondence between the intent and the target control as an intent control fingerprint; when executing the business process to be verified, match the controls of the current interface with the intent control fingerprints, and if the match is successful, use the matched control as the target control; if the match fails, input the control tree and the screenshot of the interface into the page semantic alignment model to obtain the target control, and update the intent control fingerprint with the target control.

[0017] In an exemplary embodiment of this application, when the determination module outputs the contract misalignment type, it is configured to: determine the contract misalignment type as front-end performance passed and server rejection when the interface satisfaction result is satisfied and the contract satisfaction result is not satisfied; determine the contract misalignment type as server processing and front-end not reflecting when the interface satisfaction result is not satisfied and the contract satisfaction result is satisfied; and extract the intent subsequence that causes the current intent to be executed to fail and the field based on the first evidence and the second evidence, as a reproduction slice output.

[0018] The mobile application business process verification method and system provided in this application declare the process to be verified in the form of business semantic intent, and locate the target control by using the control tree and interface screenshots through the page semantic alignment model, so that the positioning of the target control does not depend on the resource identifier and coordinates of the control. When the control identifier, coordinates or text changes under different builds, different traffic distributions, different languages ​​or different models of the mobile application interface under test, the intent sequence itself does not change accordingly, and the positioning can still fall on the control corresponding to the intent semantics, so there is no need to rewrite the verification script for each version. In this application embodiment, within the time window of executing the operation corresponding to each intent, the interface state change and the server response and data status fields triggered by the operation are jointly collected at the granularity of the intent, and the interface meets the result and the contract meets the result simultaneously before the intent is judged to pass. Thus, the business failure that the front end has displayed as successful and redirected, but the server actually rejects is judged as failing and is classified as a contract misalignment type where the front end appears to pass but the server rejects. Ultimately, regression verification allows the reuse of the same intent sequence without rewriting it version by version after the interface redesign. Business failures that are rejected by the server but have already been redirected by the front end are no longer ignored, but are intercepted, attributed, and located using reproducible slices, so that the verification conclusion of the mobile application business process is consistent with the actual business result of the process.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below.

[0021] Figure 1 A schematic diagram of the architecture of a mobile application business process verification system provided in an embodiment of this application; Figure 2 A flowchart illustrating the verification method for mobile application business processes provided in this application embodiment; Figure 3This is a schematic diagram illustrating the training process of the page semantic alignment model provided in this application embodiment. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some embodiments of this application, and not all embodiments. The terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. The content described below using directional terms is based on the accompanying drawings and is only for the convenience of describing the embodiments of this application, and should not be construed as limiting this application.

[0023] See Figure 1 , Figure 1 This is a schematic diagram of an architecture for a mobile application business process verification system provided in this application embodiment. The verification system is deployed on a computing device with a processor and memory. This computing device can be a personal computer, server, cloud host, or a test workstation connected to the mobile terminal under test. The mobile application under test runs on the mobile terminal under test. The mobile terminal under test can be a smartphone, tablet computer, or in-vehicle terminal, etc., running a mobile operating system. This application embodiment does not limit the specific form of the computing device and the mobile terminal under test.

[0024] In some embodiments of this application, the verification system and the mobile terminal under test are connected via a device debugging interface. This interface is used to read the control tree of the current interface of the mobile terminal under test, capture screenshots of the interface, and issue operation commands to the mobile terminal under test. The control tree refers to a tree-like data structure where controls on the current interface of the mobile application under test are organized hierarchically. Each node in the control tree corresponds to a control, and the node records the control's role, text, reachability label, visible area in the interface, and relationship with its parent and child nodes.

[0025] An evidence collection path exists between the verification system and the server corresponding to the mobile application under test. This evidence collection path is used to acquire server response fields generated during the interaction between the mobile application under test and the server, as well as server data status fields associated with that operation. In some embodiments of this application, the evidence collection path is implemented through a network proxy deployed between the mobile terminal under test and the server. The network proxy intercepts requests sent by the mobile application under test to the server and responses returned by the server. In other embodiments of this application, the evidence collection path is implemented through embedded point reporting within the mobile application under test. In still other embodiments of this application, the evidence collection path is implemented through a test stub interface provided by the server side. This test stub interface is used to read the data status of the server after processing the operation. The above three implementation methods can be used individually or in combination, and this application does not limit their use.

[0026] See also Figure 1 The verification system logically includes an intent acquisition module 110, an intent localization module 120, an execution module 130, an evidence collection module 140, and a judgment module 150. In some embodiments of this application, the verification system also includes a gating module 160. Each of the above modules can be implemented by the processor of the computing device executing program instructions in the memory, or it can be implemented by dedicated logic circuits. Each module corresponds to a segment of processing logic in the processor or an independent processing unit. This application does not limit the physical implementation form of each module.

[0027] The intent acquisition module 110 is connected to the intent location module 120 and is used to provide the intent to be executed to the intent location module 120. The intent location module 120 is connected to the execution module 130 and is used to provide the located target control to the execution module 130. The execution module 130 is connected to the evidence collection module 140 and is used to trigger the evidence collection module 140 to collect evidence during the execution of the operation. The evidence collection module 140 is connected to the judgment module 150 and is used to provide the collected evidence to the judgment module 150. The judgment module 150 outputs the judgment result of the intent and the contract misalignment type. The gating module 160 is connected to the intent acquisition module 110 and the execution module 130 and is used to verify the entry gating before intent location and trigger the execution of the preceding intent if the gating is not achieved.

[0028] The intent localization module 120 is internally configured with a page semantic alignment model. The page semantic alignment model is a machine learning model that takes a control tree and a screenshot of the interface as input and the matching relationship between the intent and the control as output. In some embodiments of this application, the page semantic alignment model is a two-input branch neural network, where one branch performs structural encoding of the control tree and the other branch performs visual encoding of the screenshot. The encoding results of the two branches are fused and matched with the semantic representation of the intent for scoring. The training and inference processes of the page semantic alignment model will be described in detail later.

[0029] See Figure 2 , Figure 2 This is a flowchart illustrating a mobile application business process verification method provided in an embodiment of this application. The method is executed by the aforementioned verification system. For ease of understanding, a multi-step business process is used as a vehicle to describe each step of the method. It should be noted that the execution order of the steps described below is not limited by the order in which they appear in the text, unless the context explicitly indicates that the input of one step comes from the output of another step.

[0030] Step 210: Obtain the intent sequence of the business process to be verified. Each intent in the intent sequence is associated with a corresponding evidence set, which includes interface evidence and server-side contract evidence.

[0031] The business process to be verified refers to an operational flow consisting of several business steps arranged sequentially. In some embodiments of this application, the business process to be verified is an account opening process, a transfer process, an order placement process, an insurance application process, or an appointment registration process. Each business step corresponds to an intent. The intent is a declaration of the business semantics of that business step, rather than a declaration of the specific controls operated by that business step.

[0032] For example, a sequence of intents in a transfer process may include multiple intents such as "enter transfer portal," "enter recipient account," "enter transfer amount," "submit transfer," and "confirm transfer result." Each intent is named using business semantics and is not bound to any control resource identifier or coordinates. In some embodiments of this application, intents are represented using structured fields, which include an intent identifier, an action type of the intent, and a target semantic description of the intent. The action type can be clicking, inputting, selecting, or swiping. This application does not limit the specific representation of intents.

[0033] Each intent is associated with an evidence set. This evidence set includes two categories: interface evidence and server-side contract evidence. Interface evidence describes the state the interface should appear after the intent is executed. In some embodiments of this application, interface evidence is represented by a set of business semantic conditions, such as "an amount input area appears and the amount input area is editable." Server-side contract evidence describes the state the server should meet after the intent is executed. In some embodiments of this application, server-side contract evidence is represented by a set of field conditions, such as "the server's response status code is 'acceptance successful' and the server's order status field is 'created'." The evidence set is obtained along with any graph sequence as the basis for subsequent judgment.

[0034] Step 220: For the current intent to be executed, collect the control tree and screenshot of the current interface, input the control tree and the screenshot into the page semantic alignment model to obtain the target control that matches the current intent to be executed. The determination of the target control does not depend on the resource identifier and coordinates of the control.

[0035] The intent localization module 120 extracts the intent to be executed from the intent sequence. It reads the control tree of the current interface of the tested mobile terminal via the device debugging interface and captures a screenshot of the current interface. The intent localization module 120 inputs the control tree and the screenshot into a page semantic alignment model. The page semantic alignment model assigns a matching score to each candidate control in the control tree and the intent to be executed. The intent localization module 120 selects the candidate control with the highest matching score as the target control.

[0036] In some embodiments of this application, the page semantic alignment model assigns a matching score based on the business semantics of the candidate control as presented on the interface, rather than the resource identifier or coordinates of the candidate control. Even if the resource identifier, coordinates, or text of the candidate control changes due to different builds, experimental distributions, languages, or device models of the tested mobile application, as long as the business semantics carried by the candidate control on the interface align with the currently executed intent... Figure 1 Therefore, the page semantic alignment model will still give a high matching score. Thus, the location of the target control is decoupled from the control's resource identifier and coordinates.

[0037] Step 230: Execute the operation corresponding to the currently pending intent on the target control.

[0038] The execution module 130 performs the corresponding operation on the target control based on the action type of the intent to be executed. When the action type is click, the execution module 130 initiates a click operation within the visible area of ​​the target control. When the action type is input, the execution module 130 writes the input content specified by the intent to the target control. When the action type is selection, the execution module 130 selects the option specified by the intent from the candidate options of the target control. While performing the operation, the execution module 130 records the execution time of the operation as the starting point of the subsequent evidence collection time window.

[0039] Step 240: Within the time window of the operation, at the granularity of the current intent to be executed, first evidence and second evidence are jointly collected. The first evidence is the state change of the current interface before and after the operation is executed. The second evidence is the server response field triggered by the operation and the server data status field associated with the operation.

[0040] The evidence collection module 140 defines an evidence collection time window starting from the moment the execution module 130 performs the operation. In some embodiments of this application, the evidence collection time window extends from the moment the operation is executed until no further changes related to the operation occur on the interface or the server. Within this time window, the evidence collection module 140 jointly collects two streams of evidence.

[0041] The first piece of evidence, or primary evidence, is the state change of the current interface before and after the operation is performed. The evidence collection module 140 collects the control tree and a screenshot of the interface before and after the operation is performed, compares the differences between the two, and obtains the interface state changes. These interface state changes include the appearance and disappearance of controls, as well as changes in the values ​​of the controls.

[0042] The second piece of evidence consists of the server-side response fields triggered by the operation and the server-side data status fields associated with the operation. The evidence collection module 140 obtains the responses corresponding to the requests sent by the mobile application under test to the server during the operation via the evidence collection path, and extracts the server-side response fields from the responses. The evidence collection module 140 also obtains the server-side data status fields associated with the operation via the evidence collection path.

[0043] The evidence collection module 140 binds the first piece of evidence and the second piece of evidence at the granularity of the current intent to be executed. That is, the first piece of evidence and the second piece of evidence are marked as belonging to the same intent to be executed, and are used as two pieces of evidence of the same intent for subsequent joint judgment.

[0044] Step 250: Compare the first evidence with the interface evidence of the current intent to be executed to obtain the interface satisfaction result; compare the second evidence with the server contract evidence of the current intent to be executed to obtain the contract satisfaction result.

[0045] The determination module 150 compares the first piece of evidence with the interface evidence associated with the currently pending intent. If the first piece of evidence satisfies all the business semantic conditions described by the interface evidence, the interface satisfaction result is "satisfied"; otherwise, the interface satisfaction result is "unsatisfied". The determination module 150 then compares the second piece of evidence with the server-side contract evidence associated with the currently pending intent. If the second piece of evidence satisfies all the field conditions described by the server-side contract evidence, the contract satisfaction result is "satisfied"; otherwise, the contract satisfaction result is "unsatisfied".

[0046] Step 260: When both the interface satisfaction result and the contract satisfaction result are satisfied, the current intent to be executed is determined to be successful; when either the interface satisfaction result or the contract satisfaction result is unsatisfied, the current intent to be executed is determined to be unsatisfied, and based on the combination of the interface satisfaction result and the contract satisfaction result, the contract misalignment type corresponding to the current intent to be executed is output.

[0047] The determination module 150 performs a joint determination on the interface satisfaction result and the contract satisfaction result. Only when both the interface satisfaction result and the contract satisfaction result are satisfied will the determination module 150 determine that the intent to be executed is successful. If either the interface satisfaction result or the contract satisfaction result is unsatisfactory, the determination module 150 determines that the intent to be executed is unsuccessful.

[0048] Let's illustrate the role of joint judgment with an example. In a typical scenario, after the tested mobile application submits a transfer, the front-end interface displays "Transfer successful" and redirects to the results page, indicating that the interface satisfies the result. However, the server actually rejects the transfer due to a failed risk control check, and the status code in the server's response field is "rejected," making the contract satisfaction result unsatisfactory. In this case, if the judgment is based solely on the interface satisfaction result, the intention would be judged as successful, thus allowing a failed business step to pass. However, the judgment module 150, because it requires both the interface satisfaction result and the contract satisfaction result to be satisfied simultaneously, judges the intention as unsatisfactory. Thus, the business failure where the front-end appears successful but the server rejects it is directly intercepted.

[0049] The determination module 150 further outputs the contract misalignment type corresponding to the intent to be executed, based on the combination of the interface satisfaction result and the contract satisfaction result. The contract misalignment type is used to describe the specific form of the intent failure.

[0050] In some embodiments of this application, the verification system executes steps 220 to 260 one by one for each intent in the intent sequence until all intents in the intent sequence have been processed, or until a certain intent is determined to fail and is configured to terminate upon encountering a failure. The verification system outputs the determination result of each intent in the intent sequence as the result of this business process verification.

[0051] The method provided in this application declares the process to be verified in the form of business semantic intent, and locates the target control by using a page semantic alignment model based on the control tree and interface screenshots, so that the positioning of the target control does not depend on the resource identifier and coordinates of the control. When the control identifier, coordinates or text changes under different builds, different traffic distributions, different languages ​​or different models of the mobile application interface under test, the intent sequence itself does not change accordingly, and the positioning can still fall on the control corresponding to the intent semantics, thus eliminating the need to rewrite the verification script version by version. At the same time, this application embodiment collects interface evidence and server-side contract evidence at the granularity of the intent within the time window of executing the operation corresponding to each intent, and requires both to be satisfied simultaneously to determine that the intent is passed, so that the business failure that passes the front end but is rejected by the server is intercepted and attributed, and the verification conclusion is consistent with the actual business result of the business process.

[0052] The page semantic alignment model plays a crucial role in intent localization by mapping intent to target controls. The training and inference phases of the page semantic alignment model are described below to enable those skilled in the art to implement the model. It should be noted that the model structure and training strategy described below are merely examples, and this application does not limit the specific network structure of the page semantic alignment model.

[0053] See Figure 3 , Figure 3 This is a schematic diagram illustrating the training process of the page semantic alignment model provided in an embodiment of this application. Figure 3 As shown, the training process of the page semantic alignment model may include steps S310 to S350.

[0054] In step S310, historical interface samples and annotations are obtained. Historical interface samples may include historical versions, different experimental distribution versions, different language versions, or mobile application interface samples under different device models. Each historical interface sample may include a corresponding control tree and an interface screenshot. The control tree is used to represent the hierarchical relationship, control role, text information, reachability labels, value range information, and visible area of ​​each control in the historical interface; the interface screenshot is used to represent the visual presentation state of the historical interface. Annotations are used to indicate the target control corresponding to a certain business intent in the historical interface sample. For example, for the intent to submit a transfer, the annotation can indicate the button control actually used to submit the transfer in the historical interface.

[0055] In step S320, the control tree is structured, and the screenshot is region-aligned. Specifically, each candidate control in the control tree can be converted into a control feature representation, which may include the control's role, hierarchical position, reachability label, value range, and structural relationship with adjacent controls. Simultaneously, based on the visible area of ​​the candidate control in the interface, the screenshot can be segmented or mapped to a region, so that each candidate control corresponds to a screenshot area. Thus, the same candidate control possesses both structural semantic information from the control tree and visual semantic information from the screenshot.

[0056] In step S330, a dual-branch encoding method is used to calculate the matching score. Specifically, the page semantic alignment model may include a control tree encoding branch and a visual encoding branch. The control tree encoding branch is used to encode the structured features of candidate controls to obtain the structural semantic representation of the candidate controls; the visual encoding branch is used to encode the screenshot area corresponding to the candidate controls to obtain the visual semantic representation of the candidate controls. Further, the structural semantic representation and the visual semantic representation can be fused to obtain the comprehensive semantic representation of the candidate controls. At the same time, the target semantic description of the intent is encoded to obtain the intent semantic representation. Subsequently, the matching score between the intent semantic representation and the comprehensive semantic representation of each candidate control is calculated. The higher the matching score, the more likely the candidate control is to be the target control corresponding to the intent.

[0057] In step S340, the matching loss is calculated and the model parameters are updated. Specifically, based on the annotations obtained in step S310, the labeled target control corresponding to the current intent is determined, and the labeled target control is used as a positive sample, while other candidate controls in the same historical interface samples are used as negative samples. During model training, the matching score between the intent and the labeled target control is increased, while the matching score between the intent and non-target controls is decreased, thereby calculating the matching loss. Then, based on this matching loss, the parameters of the page semantic alignment model are updated, allowing the model to gradually learn the semantic correspondence between business intents and target controls.

[0058] In step S350, it is determined whether the training termination condition is met. The training termination condition may include the number of training epochs reaching a preset maximum number of epochs, the matching loss converging, the target control matching accuracy on the validation set reaching a preset accuracy, or the matching accuracy on the validation set no longer improving for several consecutive epochs. If the training termination condition is not met, the process returns to step S330 to continue executing the matching score calculation, matching loss calculation, and model parameter update; if the training termination condition is met, the trained model parameters are saved for target control localization during subsequent business process verification.

[0059] The training samples are sourced from historical interface samples. Each historical interface sample includes a control tree, an interface screenshot, and a label. The label indicates which target control corresponds to a certain intent on that interface. In some embodiments of this application, the labels are obtained by manually annotating historical interfaces. In other embodiments of this application, the labels are semi-automatically generated from historical regression execution records, that is, the correspondence between intents and actual operation controls is extracted from historically manually confirmed execution records as labels. The above two sources of labels can be used alone or in combination.

[0060] The preprocessing of training samples includes two parts: control tree structuring and interface screenshot region alignment. Control tree structuring refers to converting each control in the control tree into a feature vector. The feature vector includes the control's role encoding, text semantic encoding, reachability label encoding, and normalized coordinates of the visible area. Interface screenshot region alignment refers to dividing the interface screenshot into several image blocks according to the visible area of ​​the controls, so that each control corresponds to an image block.

[0061] In some embodiments of this application, the page semantic alignment model employs a dual-input branch structure. The first branch is a control encoding branch, which encodes the feature vector sequence after the control tree is structured to obtain the structural semantic representation of each control. The second branch is a visual encoding branch, which encodes image blocks from a screenshot of the interface to obtain the visual semantic representation of each control. The model fuses the structural semantic representation and visual semantic representation of the same control to obtain a comprehensive semantic representation of that control. The model also includes an intent encoding branch, which encodes the target semantic description of the intent to obtain an intent semantic representation. The model calculates a matching score between the intent semantic representation and the comprehensive semantic representation of each control.

[0062] In some embodiments of this application, the loss function for model training employs a matching loss. This matching loss aims to maximize the matching score between the labeled target control and the intent, and minimize the matching score between the non-target control and the intent. It is understood that the matching loss can take the form of a contrastive loss or a cross-entropy loss; this application does not limit the specific form of the loss function.

[0063] The model training employs gradient descent to update model parameters. Training terminates when the matching accuracy between intent and target controls on the validation set stops increasing for several consecutive rounds, or when the preset maximum number of training rounds is reached. The page semantic alignment model is trained offline, and the trained model parameters are saved for use in the inference phase.

[0064] The inference phase occurs during the execution of the business process to be verified. The intent localization module 120 loads the parameters of the trained page semantic alignment model. The intent localization module 120 collects a control tree and a screenshot of the current interface, preprocessing them using the same preprocessing method as in the training phase. The intent localization module 120 inputs the preprocessed control tree and screenshot into the page semantic alignment model to obtain the matching score between each control and the intent to be executed, selecting the control with the highest matching score as the target control.

[0065] In some embodiments of this application, when the interface of the mobile application under test is a self-drawn canvas interface, resulting in a missing control tree or excessively coarse control granularity, the intent localization module 120 degenerates to using only the visual encoding branch and provides a matching score based on the interface screenshot. In other embodiments of this application, when the interface of the mobile application under test is composed of standard native controls, the intent localization module 120 may use only the control encoding branch and provide a matching score based on the control tree. The above two degenerate methods are optional implementations of the page semantic alignment model under different rendering technology stacks, and this application does not limit them.

[0066] In the above mainline embodiments, the intents in the intent sequence are located, operated, collected, and judged one by one. In some embodiments of this application, there are prerequisite state dependencies between the various intents of the business process to be verified, that is, whether the interface entry corresponding to the subsequent intent appears depends on the server-side data state achieved by the preceding intent. For such business processes, the above mainline embodiments may also include entry gate pre-verification.

[0067] The gating module 160 represents the business process to be verified as a directed step graph with preconditions. Each node in the directed step graph corresponds to an intent. The directed edges in the directed step graph represent the precondition dependencies between intents; that is, a directed edge from node A to node B indicates that entering the intent corresponding to node B requires first achieving the server-side data state generated by the intent corresponding to node A. Each node is labeled with the entry gate required to enter that node. The entry gate is the server-side data state that the preceding node must achieve.

[0068] For example, in a transfer process, the node corresponding to the intent to "enter the transfer portal" is marked with the entry gate "Account has completed real-name authentication". The server-side data status corresponding to this entry gate is achieved by the preceding real-name authentication intent. If the account has not yet completed real-name authentication, the transfer portal will not appear on the interface.

[0069] Before collecting the control tree and screenshot of the current interface, the gating module 160 verifies whether the entry gating of the node corresponding to the intent to be executed has been achieved. The gating module 160 reads the server-side data status through the evidence collection path and determines whether the status meets the conditions required by the entry gating. When the entry gating is achieved, the intent location module 120 continues to execute the subsequent location steps. When the entry gating is not achieved, the gating module 160 determines the missing preceding node based on the directed step graph. The missing preceding node is the preceding node that points to the current node in the directed step graph and whose corresponding server-side data status has not yet been achieved. The gating module 160 triggers the execution module 130 to execute the intent corresponding to the determined preceding node until the entry gating of the intent to be executed is achieved.

[0070] In some embodiments of this application, when determining a missing predecessor node, the gating module 160 traverses the directed step graph in reverse, searching for a predecessor node that has not yet been achieved layer by layer until it finds a predecessor node whose entry gating has been achieved, and starts to run each intention in the forward direction from that predecessor node.

[0071] Through the aforementioned entry gate pre-verification, when the interface entry for a subsequent intent fails to appear due to the incomplete fulfillment of the preceding server-side state, the gate module 160 identifies it as an entry gate failure and automatically completes the missing preceding intent, rather than treating it as a target control location failure and reporting an error. Thus, false positive results due to the failure of the preceding state are eliminated, and the verification process can still proceed stably even with strong reliance on the preceding state.

[0072] In the above main embodiment, the intent localization module 120 obtains the target control through a page semantic alignment model. In some embodiments of this application, the process of obtaining the target control may further include normalizing the candidate control into a business semantic fingerprint before comparing it with the intent.

[0073] The intent localization module 120 normalizes candidate controls in the control tree into business semantic fingerprints. The business semantic fingerprint includes the candidate control's role, reachability label, and value range. The role describes the interactive role the control plays in the interface, such as a button, input box, option, or text. The reachability label is the semantic label exposed by the control to the accessibility service. The value range describes the range of values ​​the control currently carries or can carry.

[0074] The intent localization module 120 compares the currently pending intent with the business semantic fingerprints of each candidate control, and selects the candidate control whose business semantic fingerprint matches the intent as the target control. During the comparison, the intent localization module 120 does not rely on the pixel rendering of the control, nor on the precise literal meaning of the control text, but rather on the normalized business semantic fingerprint.

[0075] For example, the same "confirm payment" intent may result in different text for controls in different languages, different pixel renderings on different device models, and different styles of controls under different experimental traffic distributions. However, all of these controls function as buttons, their reachability labels all point to the payment confirmation semantics, and their value ranges are consistent. The intent localization module 120 identifies all the controls that present different meanings but have equivalent business semantics as target controls that match the "confirm payment" intent based on business semantic fingerprint comparison.

[0076] In some embodiments of this application, the comparison of business semantic fingerprints employs weighted matching. Roles are mandatory, and candidate controls with inconsistent roles are directly excluded. Reachability tags and value ranges are used as weighting factors, and the matching degree is calculated according to preset weights. In other embodiments of this application, the comparison of value ranges allows for a certain tolerance range, ensuring that candidate controls with values ​​within the tolerance range are still considered a match. The above weighting method and tolerance range are optional implementations of business semantic fingerprint comparison, and this application does not limit their application.

[0077] By normalizing candidate controls into business semantic fingerprints independent of presentation before comparing them with intents, the differences in interface presentation caused by multiple languages, multiple device models, and experimental traffic splitting are absorbed. Controls with equivalent business semantics are no longer misjudged as mismatches due to different presentations. As a result, the stability of intent localization is improved in cross-language, cross-device, and cross-traffic scenarios.

[0078] After the above-mentioned business semantic fingerprint normalization and localization, some embodiments of this application may also include caching and mismatch fallback of intent control fingerprints to reduce the overhead of online inference and improve the reproducibility of results.

[0079] Before executing the business process to be verified, the intent localization module 120 establishes a semantic index for the current interface. Establishing the semantic index means pre-calculating the business semantic fingerprint of each control on the interface and caching the correspondence between intents and target controls as intent control fingerprints. The intent control fingerprint records the correspondence between the intent and its corresponding target control's business semantic fingerprint.

[0080] When executing the business process to be verified, the intent localization module 120 matches the controls on the current interface with intent control fingerprints. If a control matching the intent control fingerprint exists on the current interface, the match is successful, and the intent localization module 120 uses the matched control as the target control without calling the page semantic alignment model. If no control matching the intent control fingerprint exists on the current interface, the match fails, and the intent localization module 120 inputs the control tree and the interface screenshot into the page semantic alignment model to obtain the target control, and updates the intent control fingerprint with this target control.

[0081] In some embodiments of this application, the success of a match is determined based on the similarity between the business semantic fingerprint of the current interface control and the business semantic fingerprint recorded in the intent control fingerprint. A successful match is determined when the similarity is higher than a preset matching threshold. The preset matching threshold can be a relatively lenient value to improve cache hit rate, or a stricter value to reduce false matches. In some embodiments of this application, for business processes with relatively stable interfaces, a stricter preset matching threshold is used; for business processes with frequently updated interfaces, a more lenient preset matching threshold is used in conjunction with mismatch fallback.

[0082] By caching and mismatch fallback of intent control fingerprints, most intents can be located deterministically by matching cached fingerprints during execution. Only when an interface redesign causes fingerprint mismatch will the fallback proceed to online inference using the page semantic alignment model. As a result, the intent location results can be reproduced when the interface remains unchanged, and the overhead of online inference is no longer linearly related to the length of the intent sequence.

[0083] In the above main embodiment, the determination module 150 outputs a contract misalignment type when the intent fails. In some embodiments of this application, the process of outputting the contract misalignment type includes classifying the results based on the combination of interface satisfaction results and contract satisfaction results, and extracting a reproducible slice.

[0084] The determination module 150 determines the contract misalignment type based on the combination of the interface satisfaction result and the contract satisfaction result. When the interface satisfaction result is satisfied and the contract satisfaction result is not satisfied, the determination module 150 determines the contract misalignment type as "front-end performance passed, server rejection." This type corresponds to the aforementioned situation where the front-end has displayed success and redirected, but the server actually rejects. When the interface satisfaction result is not satisfied and the contract satisfaction result is satisfied, the determination module 150 determines the contract misalignment type as "server processing completed, front-end not reflected." This type corresponds to the situation where the server has correctly processed the operation, but the front-end has not redirected or refreshed.

[0085] In some embodiments of this application, for cases where entry gate control is not achieved, the determination module 150, in conjunction with the verification result of the gate control module 160, determines the contract misalignment type as state gate control failure. Thus, the form of intent failure is further subdivided into multiple types, such as front-end performance passing but server rejection, server processing but front-end not reflecting, and state gate control failure, making the specific form of failure distinguishable.

[0086] The determination module 150 further extracts the intent subsequence that causes the current intent to be executed to fail, along with the field, based on the fields in the first and second evidence that lead to non-compliance, and outputs them as a reproducible slice. The intent subsequence is the minimum set of preceding intents necessary to achieve the current intent's entry gate by tracing back from the current intent to be executed, plus the current intent itself. The field is any field in the first and second evidence that is inconsistent with the interface evidence or the server-side contract evidence.

[0087] For example, when a transfer intent is categorized as passing on the front end but being rejected by the server, the reproduction slice includes the intent subsequence of real-name authentication intent, intent to enter the transfer portal intent, intent to enter the amount intent, and intent to submit the transfer intent, as well as the fields in the server response with a rejected status code and the rejection reason fields. Testers can replay the intent subsequence and directly observe the key fields that caused the failure based on the reproduction slice, without having to replay the entire business process.

[0088] In some embodiments of this application, when reproducing a slice and extracting intent subsequences, the aforementioned directed procedural graph is used to traverse backwards from the current node, retaining only the preceding nodes upon which the current intent entry gate depends, and discarding side intents unrelated to the current failure. Thus, the number of intents contained in the reproducing slice is minimized.

[0089] By categorizing failed intents into specific contract misalignment types and using the minimum intent subsequence and key fields as reproducible slices for output, the failure patterns when business process verification fails can be distinguished, reducing the cost of failure location and reproduction.

[0090] In some embodiments of this application, the above-mentioned business semantic fingerprint normalization positioning may also include partial updates to the relationship between changes in the propagation of effects and positioning, so as to reduce the maintenance overhead after changes to the interface or server contract.

[0091] When the current interface or server-side contract evidence changes, the intent localization module 120 determines the set of intents affected by the change based on the correspondence between intents and business semantic fingerprints. The set of intents affected by the change is the set of intents whose corresponding target controls have changed in business semantic fingerprint due to this change.

[0092] The intent localization module 120 updates the target controls for intents that fail to match within the intent set affected by the change. The intent localization module 120 retains the target controls for intents outside the affected intent set unchanged. That is, this change only triggers updates to the localization relationships of affected intents; the localization relationships of unaffected intents are not rebuilt.

[0093] For example, when the tested mobile application redesigns the payment page, only changing the presentation of the payment confirmation control, the intent localization module 120 determines, based on the correspondence of business semantic fingerprints, that the affected intent set only includes intents related to the payment confirmation control, and other intents are not included in this set. The intent localization module 120 only relocates the target control and updates the intent control fingerprint for payment confirmation-related intents, while the intent control fingerprints for other intents remain unchanged.

[0094] In some embodiments of this application, when determining the set of intents affected by the change, the intent localization module 120 compares the business semantic fingerprints of the interface controls before and after the change, and includes the intents associated with controls whose business semantic fingerprints have changed into the set of intents affected by the change. In other embodiments of this application, for changes to the server-side contract, the intent localization module 120 includes the intents whose server-side contract evidence has changed into the set of intents affected by the change.

[0095] By changing the local updates of the propagation and location relationships, a change in one part of the mobile application interface or server contract under test only triggers an update of the location relationships of the affected intent set, while the location relationships of intents outside the set remain unchanged. Therefore, the maintenance overhead of the verification test cases is not related to the overall size of the graph sequence, but only to the number of intents affected by the change.

[0096] The above embodiments are described using a money transfer process as a vehicle. In some embodiments of this application, the business process to be verified is an account opening process, and the intent sequence includes filling in identity information, uploading documents, facial recognition, and submitting the account opening intention. The server-side contract evidence for submitting the account opening intention is that the server-side account opening status field shows "account opened." In other embodiments of this application, the business process to be verified is an e-commerce order placement process, and the intent sequence includes adding to the shopping cart, selecting a shipping address, selecting a payment method, and submitting the order. The server-side contract evidence for submitting the order intention is that the server-side order status field shows "created" and the inventory deduction field shows "deducted." In still other embodiments of this application, the business process to be verified is an insurance application process or an appointment registration process. All of the above business processes are characterized by multiple steps, strong pre-state dependencies, and server-side participation in the determination process, and are all applicable to the methods provided in the embodiments of this application.

[0097] In some embodiments of this application, the rendering technology stack of the mobile application under test uses native control rendering, with a complete control tree and fine control granularity, and intent localization is mainly based on control coding branches. In other embodiments of this application, hybrid rendering is used, with part of the interface using native controls and part using web page containers, and intent localization integrates control coding branches and visual coding branches. In still other embodiments of this application, canvas self-drawing rendering is used, with a missing control tree, and intent localization is mainly based on visual coding branches. Under all the above rendering technology stacks, intent localization is based on business semantics rather than control identifiers and coordinates.

[0098] In some embodiments of this application, server-side contract evidence is obtained through a network proxy. The network proxy is deployed between the tested mobile terminal and the server, intercepting requests sent by the tested mobile application to the server and responses returned by the server. In some embodiments of this application, the server-side response fields extracted by the network proxy include a status code field, a business result code field, an error reason field, a document status field, a document identifier field, and an acceptance time field. These fields constitute an example of a server-side response field, with each field corresponding to a server-side processing result.

[0099] In some embodiments of this application, server-side contract evidence is obtained through a server-side test stub interface. When communication between the mobile application under test and the server is encrypted and cannot be parsed by a network proxy, the test stub interface directly reads the server's data status field after processing the operation, using it as the server's data status field. In still other embodiments of this application, when both the network proxy and the test stub interface are unavailable, the evidence collection module 140 obtains the server's response field based on the embedded points reported within the mobile application under test. This fallback method for obtaining evidence in case of a miss ensures that server-side contract evidence can be obtained under various communication conditions.

[0100] In some embodiments of this application, the verification system handles abnormal branches during the intent execution process. When the intent localization module 120 fails to provide a target control with a matching score higher than a preset localization threshold on the current interface, the intent localization module 120 first verifies whether the entry gate of the current intent has been achieved through the gating module 160. If the entry gate has not been achieved, the preceding intent is processed as described above. If the entry gate has been achieved but localization is still not possible, the intent localization module 120 marks the intent as a localization anomaly and outputs the control tree and screenshot of the current interface for manual verification.

[0101] When the evidence collection module 140 fails to acquire the server response field within the evidence collection time window, it retryes acquiring it within a preset waiting time. If the server response field is still not acquired after the retry, the evidence collection module 140 sets the contract fulfillment result to unfulfilled and marks the contract misalignment type as no server response. When the server response field is inconsistent with the server data status field, for example, the response field shows successful acceptance while the data status field shows not created, the judgment module 150 sets the contract fulfillment result to unfulfilled and marks the contract misalignment type as inconsistent between the server response and data status. The above-mentioned handling of abnormal branches enables the verification system to provide clear judgments and attributions in cases of location failure, no server response, and inconsistent server response and data status.

[0102] The following uses a specific money transfer process as a vehicle to demonstrate the main solution and several typical auxiliary features overlaid in the same specific scenario. The combination of auxiliary features used in this end-to-end embodiment is only a typical combination and does not constitute a limitation on the scope of protection of this application.

[0103] The transfer process of a bank's mobile application is used as the business process to be verified. The verification system obtains the intent sequence of this transfer process, which includes intents such as real-name authentication, entering the transfer portal, entering the recipient account, entering the transfer amount, submitting the transfer, and confirming the transfer result. Each intent is associated with interface evidence and server-side contract evidence.

[0104] Before execution, the verification system establishes a semantic index on the transfer page, caching the correspondence between each intent and the target control's business semantic fingerprint as intent control fingerprints. The verification system represents the transfer process as a directed step graph with preconditions, and the node corresponding to the intent entering the transfer entry point is marked as indicating that the entry gate has completed real-name authentication.

[0105] During execution, for the intent to enter the transfer portal, the gating module 160 verifies the entry gate and finds that the account has not yet completed real-name authentication. Based on the directed step graph, it determines that the missing preceding node is the real-name authentication intent, triggering the execution module 130 to complete the real-name authentication intent until the entry gate is achieved. Subsequently, the intent localization module 120 compares the intent to enter the transfer portal with the business semantic fingerprint of the current interface control. If a cache hit occurs, the matched control is used as the target control, and the execution module 130 performs a click on the target control. The evidence collection module 140 jointly collects the interface evidence of the intent to enter the transfer portal and the server-side contract evidence, and the judgment module 150 determines that the intent is valid.

[0106] Upon submitting a transfer intention, the intention location module 120 locates the submit control through business semantic fingerprint matching, and the execution module 130 executes the click. After execution, the front-end interface displays "transfer successful" and redirects to the result page, thus satisfying the interface's result. The evidence collection module 140 collects the server's response fields through a network proxy and finds that the status code is "rejected" and the rejection reason field is "exceeded daily limit," thus changing the contract satisfaction result to "dissatisfied." The judgment module 150 determines that the transfer intention submission is unsuccessful, classifies the contract misalignment type as "front-end performance passed but server rejection," and extracts the intention subsequence of real-name authentication, entering the transfer portal, entering the receiving account, entering the transfer amount, and submitting the transfer, along with the rejection reason field, as a reproduction slice for output.

[0107] When the bank's mobile application redesigns the transfer page and only changes the presentation of the amount input control, the intent positioning module 120 determines, based on the correspondence of the business semantic fingerprint, that the affected intent set only includes the intent to input the transfer amount, and only repositions and updates the intent control fingerprint for that intent, while the positioning relationships of other intents remain unchanged.

[0108] This end-to-end implementation superimposes multiple features such as entry gate pre-verification, business semantic fingerprint normalization positioning, intent control fingerprint caching and mismatch fallback, contract misalignment attribution and reproduction slicing, change impact propagation and positioning relationship self-healing, presenting the synergy between the main scheme and typical subordinate features in the same specific scenario.

[0109] The intent determination results, contract misalignment types, and reproduction slices output by the verification system are reused in subsequent business processes in some embodiments of this application. In some embodiments of this application, the intent determination results are summarized into a regression verification report of the business process for use in version release decisions. In other embodiments of this application, the contract misalignment types and reproduction slices are pushed to the defect management system, automatically creating defect tickets with reproduction steps. The solutions provided in the embodiments of this application do not depend on the specific form of the aforementioned subsequent business processes; the reuse methods of the aforementioned subsequent business processes are merely examples.

[0110] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the steps of the verification method for the mobile application business process described above. The computer-readable storage medium may be a read-only memory, random access memory, magnetic disk, or optical disk, or other media capable of storing program code.

[0111] This application also provides an electronic device, which includes a processor and a memory. The memory stores a computer program that can run on the processor. When the computer program is executed by the processor, it implements the steps of the verification method for the mobile application business process described above. It is understood that the functions corresponding to the above modules can be implemented by the processor executing the program instructions in the memory.

[0112] In the embodiments of this application, the orientations or positional relationships indicated by terms such as "center," "longitudinal," "lateral," "up," "down," "front," "rear," "left," "right," "vertical," and "horizontal" are based on the orientations or positional relationships shown in the accompanying drawings. They are used solely for the convenience of describing the embodiments of this application and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. "Multiple" means at least two, unless otherwise explicitly defined. "Connected" and "linked" should be interpreted broadly.

[0113] The embodiments described above are merely illustrative of the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application. The dimensions of the parts in the accompanying drawings are independent of the actual object, and the actual object dimensions can be arbitrarily changed.

[0114] This application involves several thresholds, including a preset positioning threshold for intent localization, a preset matching threshold for intent control fingerprint matching, and a tolerance range for the median range of business semantic fingerprint comparison. The numerical ranges and scenario-specific value refinements for these thresholds are given below to support the implementation of the corresponding features.

[0115] A preset location threshold is used to determine whether the matching score given by the page semantic alignment model is sufficient to identify the candidate control as the target control. In some embodiments of this application, the matching score is normalized to the range of 0 to 1, and the preset location threshold ranges from 0.5 to 0.95. In scenarios where the interface controls of the tested mobile application have high distinguishability and few similar controls, the preset location threshold can be set to a lower sub-range, such as 0.5 to 0.65, to improve the recall of the location. In scenarios where there are multiple controls with similar business semantics on the interface and false location is prone to occur, the preset location threshold can be set to a higher sub-range, such as 0.8 to 0.95, to reduce false location. In general scenarios, the preset location threshold can be set to a middle sub-range, such as 0.65 to 0.8. The above sub-ranges aim to balance the recall and false location rate of the location, and different sub-ranges correspond to different balance preferences.

[0116] A preset matching threshold is used to determine whether the business semantic fingerprint of the current interface control matches the business semantic fingerprint recorded in the intent control fingerprint. In some embodiments of this application, the preset matching threshold ranges from 0.6 to 0.98. For business processes with relatively stable interfaces and low update frequency, a higher preset matching threshold, such as 0.9 to 0.98, can be used to improve the reliability of cache hits and make deterministic matching as accurate as possible. For business processes with frequent interface updates, a lower preset matching threshold, such as 0.6 to 0.75, can be used to improve cache hit rate and cooperate with mismatch rollback online inference. In scenarios with moderate update frequency, a middle preset matching threshold, such as 0.75 to 0.9, can be used. The above grading aims to optimize the balance between cache hit rate and deterministic matching accuracy.

[0117] The tolerance range of the value range in business semantic fingerprint comparison is used to absorb slight differences in the value range of the same business semantic under different presentations. In some embodiments of this application, for numerical value ranges, the tolerance range is a certain proportion of the width of the value range, ranging from 0 to 0.2. In scenarios where high accuracy of value acquisition is required, the tolerance range can be a smaller sub-range, such as 0 to 0.05; in scenarios where multiple models and multiple languages ​​lead to large differences in the value range presentation, the tolerance range can be a larger sub-range, such as 0.1 to 0.2. The above-mentioned tolerance range sub-ranges are optimized to balance the tolerance of value range matching with the mismatch rate.

[0118] In some embodiments of this application, the values ​​of the aforementioned thresholds can be automatically adjusted by the verification system based on the interface redesign history of the mobile application under test. For example, when the verification system detects an increase in the fingerprint mismatch frequency of a certain business process, it automatically adjusts the preset matching threshold of that business process to a lower sub-level. The above-mentioned automatic adjustment method is an optional implementation of the threshold values, and this application does not limit it.

[0119] In some embodiments of this application, the intent is represented by structured data, which includes an intent identifier field, an action type field, a target semantic description field, an interface evidence field, and a server-side contract evidence field. The intent identifier field uniquely identifies the intent. The action type field takes the value of one of click, input, selection, or swipe. The target semantic description field describes the business semantics of the control to be manipulated by the intent. The interface evidence field records the business semantic conditions that the interface should meet after the intent is executed. The server-side contract evidence field records the field conditions that the server should meet after the intent is executed. These fields constitute a field example of an intent, with each field corresponding to an attribute of the intent.

[0120] In some embodiments of this application, server-side contract evidence is represented as a set of conditional expressions. Each conditional expression includes a server-side field name, a comparison operator, and an expected value. For example, a conditional expression could be "the server-side order status field equals 'created'" or "the server-side inventory deduction field equals 'deducted'". When comparing the contract fulfillment result, the judgment module 150 substitutes the collected server-side response field and server-side data status field into each conditional expression to evaluate. The contract fulfillment result is considered satisfied when all conditional expressions are true. The above form of conditional expressions allows server-side contract evidence to be configured by business personnel and can cover various server-side fields such as status codes, business result codes, and document status.

[0121] By representing intents and evidence sets with structured data and conditional expressions, intent sequences and evidence sets can be configured and maintained independently, and can cover different requirements of different business processes for interface and server states, making the method provided in this application applicable to mobile application verification in a variety of complex business scenarios.

Claims

1. A method for verifying a mobile application business process, characterized in that, include: Obtain the intent sequence of the business process to be verified. Each intent in the intent sequence is associated with a corresponding evidence set, which includes interface evidence and server-side contract evidence. For the intent to be executed, the control tree and screenshot of the current interface are collected. The control tree and the screenshot are input into the page semantic alignment model to obtain the target control that matches the intent to be executed. The determination of the target control does not depend on the resource identifier and coordinates of the control. Execute the operation corresponding to the currently pending intent on the target control; Within the time window of the operation, the first evidence and the second evidence are jointly collected at the granularity of the current intent to be executed. The first evidence is the state change of the current interface before and after the operation is executed, and the second evidence is the server response field triggered by the operation and the server data status field associated with the operation. The first piece of evidence is compared with the interface evidence of the current intent to be executed to obtain an interface satisfaction result; the second piece of evidence is compared with the server-side contract evidence of the current intent to be executed to obtain a contract satisfaction result. When both the interface satisfaction result and the contract satisfaction result are satisfied, the current intent to be executed is determined to be successful; when either the interface satisfaction result or the contract satisfaction result is unsatisfied, the current intent to be executed is determined to be unsatisfied, and based on the combination of the interface satisfaction result and the contract satisfaction result, the contract misalignment type corresponding to the current intent to be executed is output.

2. The method according to claim 1, characterized in that, After obtaining the intent sequence of the business process to be verified, the method further includes: The business process to be verified is represented as a directed step graph with preconditions. Each node in the directed step graph corresponds to an intent. Each node is marked with an entry gate required to enter the node. The entry gate is the server-side data state that the preceding node should achieve. Before collecting the control tree and screenshot of the current interface, verify whether the entry gate of the node corresponding to the intent to be executed has been achieved; When the entry gate is not achieved, the missing preceding node is determined according to the directed step graph, and the intent corresponding to the preceding node is executed until the entry gate is achieved.

3. The method according to claim 1, characterized in that, The process of obtaining the target control that matches the currently pending intent includes: The candidate controls in the control tree are normalized into business semantic fingerprints, which include the role, reachability label and value range of the candidate controls. The current intent to be executed is compared with the business semantic fingerprint, and the candidate control whose business semantic fingerprint matches the intent is taken as the target control.

4. The method according to claim 3, characterized in that, After normalizing the candidate controls into business semantic fingerprints, the method further includes: Before executing the business process to be verified, a semantic index is established for the current interface, and the correspondence between the intent and the target control is cached as an intent control fingerprint. When executing the business process to be verified, the controls on the current interface are matched with the intent control fingerprint; When a match is successful, the matched control is used as the target control; when a match fails, the control tree and the screenshot of the interface are input into the page semantic alignment model to obtain the target control, and the intent control fingerprint is updated with the target control.

5. The method according to claim 1, characterized in that, The output of the contract misalignment type corresponding to the currently pending intent includes: When the interface satisfies the result and the contract satisfies the result, the contract misalignment type is determined to be that the front-end performance is approved and the server rejects it. When the interface satisfies the result of not satisfying the contract and the contract satisfies the result of satisfying the contract, the contract misalignment type is determined to be that the server has processed it but the front end has not reflected it. Based on the fields in the first and second evidence that cause non-compliance, extract the intention subsequence that causes the current intention to be executed to fail, along with the fields, and output them as a reproduction slice.

6. The method according to claim 3, characterized in that, Also includes: When the current interface or the server-side contract evidence changes, the set of intents affected by the change is determined based on the correspondence between the intent and the business semantic fingerprint. For intents that fail to match in the intent set, update their corresponding target controls, while leaving the target controls for intents outside the intent set unchanged.

7. A verification system for mobile application business processes, characterized in that, include: The intent acquisition module is used to acquire the intent sequence of the business process to be verified. Each intent in the intent sequence is associated with a corresponding evidence set, which includes interface evidence and server-side contract evidence. The intent localization module is used to collect the control tree and screenshot of the current interface for the intent to be executed, input the control tree and the screenshot into the page semantic alignment model, and obtain the target control that matches the intent to be executed. The determination of the target control does not depend on the resource identifier and coordinates of the control. An execution module is used to perform the operation corresponding to the currently pending intent on the target control; The evidence collection module is used to jointly collect first evidence and second evidence within the time window of the operation, with the current intention to be executed as the granularity. The first evidence is the state change of the current interface before and after the operation is executed, and the second evidence is the server response field triggered by the operation and the server data status field associated with the operation. The determination module is used to compare the first evidence with the interface evidence to obtain an interface satisfaction result, compare the second evidence with the server contract evidence to obtain a contract satisfaction result, determine that the current intent to be executed is passed when both the interface satisfaction result and the contract satisfaction result are satisfied, determine that the current intent to be executed is not passed when there is a non-satisfaction, and output the contract misalignment type corresponding to the current intent to be executed based on the combination of the interface satisfaction result and the contract satisfaction result.

8. The system according to claim 7, characterized in that, It also includes a gating module; The gating module is used to represent the business process to be verified as a directed step graph with preconditions. Each node in the directed step graph corresponds to an intent. The node is marked with the entry gate required to enter the node. The entry gate is the server-side data state that the preceding node should achieve. The gating module is also used to verify whether the entry gating of the node corresponding to the intent to be executed is achieved before the intent positioning module collects the control tree and screenshot of the current interface. If the entry gating is not achieved, the module determines the missing preceding node according to the directed step graph and triggers the execution module to execute the intent corresponding to the preceding node until the entry gating is achieved.

9. The system according to claim 7, characterized in that, The intent localization module is also used for: Before executing the business process to be verified, a semantic index is established for the current interface, and the correspondence between the intent and the target control is cached as an intent control fingerprint. When executing the business process to be verified, the controls on the current interface are matched with the intent control fingerprint. If the match is successful, the matched control is used as the target control. If the match fails, the control tree and the screenshot of the interface are input into the page semantic alignment model to obtain the target control, and the intent control fingerprint is updated with the target control.

10. The system according to claim 7, characterized in that, When the determination module outputs the contract misalignment type, it is used for: When the interface satisfies the result and the contract satisfies the result, the contract misalignment type is determined to be that the front-end performance is approved and the server rejects it. When the interface satisfies the condition but the contract satisfies the condition, the contract misalignment type is determined to be that the server has processed the issue but the front end has not reflected it. Based on the fields in the first and second evidence that cause non-compliance, extract the intention subsequence that causes the current intention to be executed to fail, along with the fields, and output them as a reproduction slice.

Images