UI automatic acceptance method

Abstract

The invention belongs to the technical field of automatic testing, and discloses a UI automatic acceptance method. S1, the UI automatically accepts the whole process; after a native end user selects an acceptance area, control information of the native end user is sent to the server end, the server end starts a Web file exported by Sketch and extracts the control information by using a JS script, and then the control information is compared with the information sent by the native end so as to confirm a control matched with the selected area in a design draft. By automatically executing the test cases, the time and manpower resources required by manual testing are greatly reduced, so that the testing efficiency is improved, meanwhile, the possibility of human errors is reduced through automatic testing, accurate execution of testing steps and consistent result output are ensured, the testing accuracy is remarkably improved, wide test cases can be covered, and the testing efficiency is improved. Comprising conventional and edge conditions, and the stability and correctness of the application program in various scenes are ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of automated testing, and in particular to a UI automated acceptance method. Background Art

[0002] In the field of modern software development, UI automated acceptance testing is a key link in ensuring application quality and user experience. In view of the low efficiency and frequent human errors of traditional manual testing, AI image comparison algorithm, as an emerging solution for UI automated acceptance, is gradually gaining favor in the industry. This method automatically captures screenshots and uses AI image recognition technology to compare old and new screenshots to accurately identify interface changes and differences, significantly improving the efficiency and accuracy of acceptance testing. Its advantages lie in its strong scalability, ability to efficiently process a large number of screenshots, and reduction of manual intervention, ensuring the standardization and consistency of the acceptance process. However, the AI ​​image comparison method is not perfect either. The main challenges it faces include: the subjectivity of image difference judgment may lead to misjudgment; it is difficult to capture the subtle changes of dynamic and interactive elements; changes; the difficulty of balancing algorithm sensitivity and tolerance, which requires fine-tuning to avoid overly strict or loose judgment criteria; specific environmental factors such as lighting and resolution may affect recognition accuracy; the challenges of complex UI scenarios and changing states, such as element stacking and occlusion, increase the risk of algorithm misjudgment; and the high development and maintenance costs brought about by highly customized requirements. Therefore, when using the AI ​​image comparison method for UI automation acceptance, its advantages and potential disadvantages must be fully considered. The project team needs to evaluate factors such as environmental complexity, change frequency, and resource investment based on actual needs, and make wise choices after weighing the pros and cons. At the same time, the algorithm should be continuously optimized, parameters adjusted, and manual review should be combined to ensure the comprehensiveness and accuracy of the acceptance test, ultimately ensuring the high-quality delivery of software applications. Summary of the invention

[0003] The purpose of the present invention is to provide a UI automation acceptance method to solve the problems raised in the above background technology.

[0004] In order to achieve the above object, the present invention provides the following technical solution: UI automation acceptance method, the specific steps of the acceptance method include:

[0005] S1, the overall process of UI automation acceptance; after the native end user selects the acceptance area, its control information is sent to the server, which starts the Web file exported by Sketch and uses JS script to extract the control information, and then compares it with the information sent by the native end to confirm the controls in the design draft that match the selected area, completing the control comparison process;

[0006] S2, control matching algorithm; the cross-end control matching strategy involves comparing control width and type, and implementing row arrangement rules to locate controls, while considering reasonable errors and difference values. Ultimately, users are required to confirm the most reasonable matching element;

[0007] S3, control comparison algorithm; extracts sub-control information from the design draft and the actual development page, compares text content, color, font and other attributes, sets thresholds to evaluate differences, and generates a detailed sub-control comparison report to reveal inconsistencies.

[0008] Preferably, in said S1;

[0009] A1. Select the acceptance area on the native side: In the native application, the user needs to select the area for acceptance;

[0010] A2. Sending the control information data of the selected area to the server: Once the user completes the selection of the acceptance area, the native end sends the control information data of the selected area to the server;

[0011] A3 The server starts a web file exported from Sketch: The server starts a web file exported from Sketch, which contains a static display of the design draft;

[0012] A4. Use JS script to extract Web control information: In the server environment, use JavaScript script to extract the information of each control in the design draft from the Web file;

[0013] A5. Compare Web control information and client selected UI information: Once the server extracts the control information in the Web file and the control information data of the selected area sent by the native end, a comparison operation is performed; the comparison is performed based on the location, type, and attributes of the control;

[0014] A6. Control comparison completed: By comparing the Web control information and the control information of the selected area on the native side, confirm the specific control in the design draft that matches the selected area in the native application.

[0015] Preferably, the detailed steps of the cross-end control matching strategy in the S2 control matching algorithm include:

[0016] B1. Compare control widths: To ensure consistency between the design draft (exported as an H5 webpage) and the actual development page (native development), you need to compare their control widths;

[0017] B2. Compare control types: Assume that the design draft image and the actual development page use the same UI control library, so in the comparison, the consistency is determined by comparing the control types;

[0018] B3. Arrange rows and implement a row arrangement strategy based on the arrangement rules.

[0019] Preferably, the specific rules in the arrangement rules in B3 are as follows:

[0020] C1, find the element closest to the upper left corner of the parent view as the starting element A;

[0021] Then, find all elements in the same row as element A and form the first row of the array [[A,B,C]];

[0022] C2, check the element with the largest Y+Height value in the first row of the array, assuming it is element D;

[0023] C1, calculate the remaining elements that are not added to the first row and are in the same row as element D, and add them to the second row of the array;

[0024] C3, when element E and element D are in the same row, the second row array is [[A, B, C, D], [E]]; continue to check the remaining elements that have not been added to the array, determine whether they are in the same row as the elements in the second row array, and add them to the corresponding row array;

[0025] C4, and so on, until all elements are added to the corresponding row array positions;

[0026] C5, during the row arrangement process, it is important to note that the element types added to the row array can be 'text', 'img', 'other', and should be classified according to actual conditions.

[0027] Preferably, the arrangement strategy determines the positional relationship of the elements in the parent view and arranges them row by row; after the above main algorithm filtering, the remaining controls may have the same type, and at this time, it needs to be returned to the client user for confirmation in order to confirm the most reasonable element

[0028] Preferably, both the client and the Web end need to calculate all controls based on the upper left corner position of the parent view according to the absolute layout; the advantage of the absolute layout is that the calculation rules of all ends are unified and the relative positions are unified; the order of output after the calculation rows are arranged according to the above figure is:

[0029] [['A','B'],['C','D'],['E'],['F','G']].

[0030] Preferably, the main calculation process in the S3 control comparison algorithm includes:

[0031] D1. Get sub-control information: extract sub-control related information from the matching controls in the design draft and the actual development page;

[0032] D2. Compare text contents: For text sub-controls, compare whether their text contents are consistent or similar;

[0033] D3. Compare other attributes: When the child control has other attributes, compare whether the values ​​of color and font attributes are consistent;

[0034] D4. Set threshold or tolerance: For each attribute comparison, set the threshold and tolerance according to the actual situation;

[0035] D5. Generate sub-control comparison report: Based on the comparison results, generate a sub-control comparison report to show the differences between the design draft and the actual development page.

[0036] Preferably, in the comparison of control types, by comparing the width and type of controls, a certain area is selected in the native development page, and the corresponding control is found in the H5 version of the design draft. Note that when comparing controls, a reasonable error range and difference value are taken into account.

[0037] The beneficial effects of the present invention are as follows:

[0038] 1. The present invention greatly reduces the time and human resources required for manual testing by automating the execution of test cases, thereby improving testing efficiency. At the same time, automated testing reduces the possibility of human errors, ensures the precise execution of test steps and consistent result output, and significantly improves the accuracy of testing.

[0039] 2. The present invention can cover a wide range of test cases, including routine and edge cases, ensuring the stability and correctness of the application in various scenarios. In addition, the automated test cases can be executed repeatedly, enhancing the repeatability of the test, which is crucial for the continuous integration and continuous delivery processes, ensuring reliable automated acceptance for each build.

[0040] 3. The present invention provides a shared test asset library, which promotes cooperation and communication among team members and helps to better manage and track test progress. At the same time, although the initial investment is high, in the long run, automated testing can save a lot of human resources and time, reduce the workload of manual testing, and thus reduce the overall testing cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is the overall logic flow chart of the present invention;

[0042] Figure 2 It is a client user confirmation diagram of the present invention;

[0043] Figure 3 Source code screenshots for obtaining control types in key source codes:

[0044] Figure 4 To obtain the source code screenshot of the closest control.

[0045] In the picture: A, name of 4S store; B, selling price; C, sales area; D, preferential policies; E, address of 4S store; F, contact information; G, explanation of battery life. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] like Figure 1 to Figure 2 As shown, the embodiment of the present invention provides a UI automation acceptance method, and the specific steps of the acceptance method include:

[0048] S1, the overall process of UI automation acceptance; after the native end user selects the acceptance area, its control information is sent to the server, which starts the Web file exported by Sketch and uses JS script to extract the control information, and then compares it with the information sent by the native end to confirm the controls in the design draft that match the selected area, completing the control comparison process;

[0049] S2, control matching algorithm; the cross-end control matching strategy involves comparing control width and type, and implementing row arrangement rules to locate controls, while considering reasonable errors and difference values. Ultimately, users are required to confirm the most reasonable matching element;

[0050] S3, control comparison algorithm; extracts sub-control information from the design draft and the actual development page, compares text content, color, font and other attributes, sets thresholds to evaluate differences, and generates a detailed sub-control comparison report to reveal inconsistencies.

[0051] Among them, S1;

[0052] A1. Select the acceptance area on the native side: In the native application, the user needs to select the area that needs to be accepted. This can be achieved by the user selecting the area on the application interface.

[0053] A2. Sending the control information data of the selected area to the server: Once the user completes the selection of the acceptance area, the native end sends the control information data of the selected area to the server; the data may include the location, size, type and other related information of the control;

[0054] A3 The server starts a web file exported from Sketch: The server starts a web file exported from Sketch, which contains a static display of the design draft;

[0055] A4. Use JS script to extract Web control information: In the server environment, use JavaScript script to extract the information of each control in the design draft from the Web file; the information may include the location, size, type, style, etc. of the control;

[0056] A5. Compare Web control information with client selected UI information: Once the server extracts the control information in the Web file and the control information data of the selected area sent by the native end, a comparison operation is performed; the comparison is performed based on the location, type, and attributes of the control to determine the specific corresponding control in the Web file;

[0057] A6. Control comparison completed: By comparing the Web control information with the control information of the selected area on the native side, the specific control in the design draft that matches the selected area in the native application is confirmed; thus completing the control comparison process.

[0058] Among them, the detailed steps of the cross-end control matching strategy in the S2 control matching algorithm include:

[0059] B1. Compare control widths: To ensure consistency between the design draft (exported as an H5 webpage) and the actual development page (native development), their control widths need to be compared; the actual development page may have certain UI errors, so the overall width may have certain differences; during the comparison process, a width range can be set to allow a certain error to determine the width difference within a reasonable range;

[0060] B2. Compare control types: Assuming that the design draft image and the actual development page use the same UI control library, the consistency is determined by comparing the control types. Generally speaking, the number of controls in the design draft and the actual development page should be consistent. However, there may be some special cases, such as the text content may not be completely consistent. During the comparison process, a difference value can be set to determine the reasonable difference in the number of texts.

[0061] B3. Arrange rows and implement a row arrangement strategy based on the arrangement rules.

[0062] Among them, the specific rules in B3 according to the arrangement rules are as follows;

[0063] C1, find the element closest to the upper left corner of the parent view as the starting element A;

[0064] Then, find all elements in the same row as element A and form the first row of the array [[A,B,C]];

[0065] C2, check the element with the largest Y+Height value in the first row of the array, assuming it is element D;

[0066] C1, calculate the remaining elements that are not added to the first row and are in the same row as element D, and add them to the second row of the array;

[0067] C3, when element E and element D are in the same row, the second row array is [[A, B, C, D], [E]]; continue to check the remaining elements that have not been added to the array, determine whether they are in the same row as the elements in the second row array, and add them to the corresponding row array;

[0068] C4, and so on, until all elements are added to the corresponding row array positions;

[0069] C5, during the row arrangement process, it is important to note that the element types added to the row array can be 'text', 'img', 'other', and should be classified according to actual conditions.

[0070] Among them, the arrangement strategy determines the position relationship of the elements in the parent view and arranges them in rows; the row arrangement method better organizes and lays out the elements to achieve a more accurate and consistent interface display; after the above main algorithm filtering, the remaining controls may have the same type. At this time, it needs to be returned to the client user for confirmation in order to confirm the most reasonable element

[0071] Among them, both the client and the Web end need to be based on the upper left corner position of the parent view, and all controls are calculated according to the absolute layout; the advantage of the absolute layout is that the calculation rules and relative positions of all ends are unified; the order of output after the calculation rows are arranged according to the above figure is:

[0072] [['A','B'],['C','D'],['E'],['F','G']].

[0073] Among them, the main calculation process in the S3 control comparison algorithm includes:

[0074] D1. Get sub-control information: Extract relevant information of sub-controls from the matching controls in the design draft and the actual development page; including the location, size, text content, color, font and other properties of sub-controls;

[0075] D2. Compare text content: For text sub-controls, compare whether their text contents are consistent or similar; use string matching algorithms (such as Levenshte in distance) and similarity algorithms (such as cosine similarity) to evaluate the degree of difference in text content;

[0076] D3. Compare other attributes: When the child control has other attributes, compare the values ​​of color and font attributes to see if they are consistent; the degree of difference between them can be evaluated by comparing the numerical values ​​of the attribute values ​​or using a similarity algorithm;

[0077] D4. Set thresholds or tolerances: For each attribute comparison, set thresholds and tolerances according to the actual situation. For example, for text content comparison, you can set a maximum edit distance or similarity threshold, and only differences within this threshold are considered consistent.

[0078] D5. Generate sub-control comparison report: Based on the comparison results, generate a sub-control comparison report to show the differences between the design draft and the actual development page; the report can include a list of mismatched sub-controls, as well as a detailed description of their differences and inconsistencies.

[0079] Among them, in the comparison of control types, by comparing the control width and type, you can select a certain area in the native development page and find the corresponding control in the H5 version of the design draft. Note that when comparing controls, consider a reasonable error range and difference value to ensure a certain compatibility and flexibility between the design draft and the actual development page; this way, you can find the corresponding control more accurately and take screenshots.

[0080] Among them, the key source code obtains the control type such as Figure 3 As shown:

[0081] Get the closest control such as Figure 4 As shown:

[0082] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0083] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. UI automation acceptance method, characterized by: The specific steps of the acceptance method include: S1, the overall process of UI automation acceptance; after the native end user selects the acceptance area, its control information is sent to the server, which starts the Web file exported by Sketch and uses JS script to extract the control information, and then compares it with the information sent by the native end to confirm the controls in the design draft that match the selected area, completing the control comparison process; S2, control matching algorithm; the cross-end control matching strategy involves comparing control width and type, and implementing row arrangement rules to locate controls, while considering reasonable errors and difference values. Ultimately, users are required to confirm the most reasonable matching element; S3, control comparison algorithm; extracts sub-control information from the design draft and the actual development page, compares text content, color, and font attributes, sets thresholds to evaluate differences, and generates a detailed sub-control comparison report to reveal inconsistencies.

2. The UI automation acceptance method according to claim 1, characterized in that: In said S1; A1. Select the acceptance area on the native side: In the native application, the user needs to select the area for acceptance; A2. Sending the control information data of the selected area to the server: Once the user completes the selection of the acceptance area, the native end sends the control information data of the selected area to the server; A3 The server starts a web file exported from Sketch: The server starts a web file exported from Sketch, which contains a static display of the design draft; A4. Use JS script to extract Web control information: In the server environment, use JavaScript script to extract the information of each control in the design draft from the Web file; A5. Compare Web control information and client selected UI information: Once the server extracts the control information in the Web file and the control information data of the selected area sent by the native end, a comparison operation is performed; the comparison is performed based on the location, type, and attributes of the control; A6. Control comparison completed: By comparing the Web control information and the control information of the selected area on the native side, confirm the specific control in the design draft that matches the selected area in the native application.

3. The UI automation acceptance method according to claim 1, characterized in that: The detailed steps of the cross-end control matching strategy in the S2 control matching algorithm include: B1. Compare control widths: To ensure consistency between the design draft (exported as an H5 webpage) and the actual development page (native development), you need to compare their control widths; B2. Compare control types: Assume that the design draft image and the actual development page use the same UI control library, so in the comparison, the consistency is determined by comparing the control types; B3. Arrange rows and implement a row arrangement strategy based on the arrangement rules.

4. The UI automation acceptance method according to claim 3, characterized in that: The specific rules in B3 according to the arrangement rules are as follows: C1, find the element closest to the upper left corner of the parent view as the starting element A; Then, find all elements in the same row as element A and form the first row of the array [[A,B,C]]; C2, check the element with the largest Y+Height value in the first row of the array, assuming it is element D; C1, calculate the remaining elements that are not added to the first row and are in the same row as element D, and add them to the second row of the array; C3, when element E and element D are in the same row, the second row array is [[A, B, C, D], [E]]; continue to check the remaining elements that have not been added to the array, determine whether they are in the same row as the elements in the second row array, and add them to the corresponding row array; C4, and so on, until all elements are added to the corresponding row array positions; C5, during the row arrangement process, it is necessary to pay attention to the types of elements added to the row array: 'text', 'img', 'other', and classify them according to the actual situation.

5. The UI automation acceptance method according to claim 3, characterized in that: The arrangement strategy determines the positional relationship of the elements in the parent view and arranges them row by row. After the above main algorithm filtering, the remaining controls may have the same type. At this time, it needs to be returned to the client user for confirmation in order to confirm the most reasonable element.

6. The UI automation acceptance method according to claim 4, characterized in that: Both the client and the Web end need to calculate all controls based on the upper left corner of the parent view according to the absolute layout. The advantage of the absolute layout is that the calculation rules and relative positions of all ends are unified. The order of output after the calculation rows are arranged in the above figure is: [['A','B'],['C','D'],['E'],['F','G']].

7. The UI automation acceptance method according to claim 1, characterized in that: The main calculation process in the S3 control comparison algorithm includes: D1. Get sub-control information: extract sub-control related information from the matching controls in the design draft and the actual development page; D2. Compare text contents: For text sub-controls, compare whether their text contents are consistent or similar; D3. Compare other attributes: When the child control has other attributes, compare the values ​​of color and font attributes to see if they are consistent; D4. Set threshold or tolerance: For each attribute comparison, set the threshold and tolerance according to the actual situation; D5. Generate sub-control comparison report: Based on the comparison results, generate a sub-control comparison report to show the differences between the design draft and the actual development page.

8. The UI automation acceptance method according to claim 3, characterized in that: In the comparison control type, by comparing the width and type of the control, it is possible to select a certain area in the native development page and find the corresponding control in the H5 version of the design draft. Note that when comparing controls, a reasonable error range and difference value should be considered.

Images