User interface test method and device, electronic equipment and storage medium

Through automated testing methods and pre-trained neural network model to detect interface elements, the problems of low efficiency and poor accuracy of user interface testing in the prior art are solved, and automatic adaptability testing of multiple device parameters is realized, and the testing efficiency and accuracy.

CN119088672APending Publication Date: 2024-12-06GREE ELECTRIC APPLIANCE INC OF ZHUHAI +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410984142.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-22
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Existing user interface testing methods rely on manual operation and visual inspection, are inefficient and error-prone, making it difficult to adapt to different screen sizes, resolutions and operating systems.

Method used

The automated testing method is adopted to deploy the test environment, obtain the device parameters of the simulated device and the pre-layout user interface, output the display interface through the automated test case output, and use the pre-trained neural network model to detect whether the interface elements overflow and determine the interface adaptability.

Benefits of technology

Reduces labor costs, improves testing efficiency and accuracy, and can automatically test the user interface's adaptability in a variety of screen sizes, resolutions and operating systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119088672A_ABST
    Figure CN119088672A_ABST
Patent Text Reader

Abstract

The embodiment of the invention provides a user interface test method and device, electronic equipment and a storage medium, and the method comprises the steps: deploying a test environment, and obtaining an automatic test case; acquiring equipment parameters of the simulation equipment and a pre-arranged user interface; the equipment parameters comprise various screen sizes, various resolutions and various operating systems; executing the automatic test case in the test environment, and outputting display interfaces of a pre-arranged user interface in various screen sizes, various resolutions and various operating systems; and according to the display interface, determining whether a pre-arranged user interface is adaptive to the equipment parameters. Therefore, whether the pre-arranged user interface is matched with various screen sizes, various resolutions and various operating systems or not is tested through the equipment parameters of the simulation equipment, meanwhile, manual testing does not need to be depended on, and through automatic testing, the labor cost is reduced, and the testing efficiency and accuracy are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of automated testing technology, and in particular to a user interface testing method, a user interface testing device, an electronic device and a computer-readable storage medium. Background Art

[0002] With the development and popularization of smart devices, the diversification of smart devices and software testing are becoming increasingly complex and sophisticated. The UI needs to adapt to different screen sizes, resolutions and operating systems. The UI (User Interface) testing methods in related technologies often need to rely on manual operation testing and manual visual inspection, which is not only inefficient but also prone to errors. Summary of the invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a user interface testing method, a user interface testing method device, an electronic device and a computer-readable storage medium that overcome the above problems or at least partially solve the above problems.

[0004] In order to solve the above problem, an embodiment of the present invention discloses a user interface testing method, the method comprising:

[0005] Deploy the test environment and obtain automated test cases;

[0006] Obtaining device parameters of a simulated device and a pre-layout user interface; the device parameters include: multiple screen sizes, multiple resolutions, and multiple operating systems;

[0007] Execute the automated test case in the test environment, and output display interfaces of the pre-layout user interface under the multiple screen sizes, the multiple resolutions, and the multiple operating systems;

[0008] According to the display interface, it is determined whether the pre-layout user interface is adapted to the device parameters.

[0009] Optionally, determining, according to the display interface, whether the pre-layout user interface is adapted to the device parameters includes:

[0010] Calling a screenshot command to take a screenshot of the display interface to obtain a first picture;

[0011] Inputting the first image into a detection model to detect whether there is element overflow at edge and contour positions in the first image; the detection model is a model obtained by pre-training a neural network model;

[0012] If it is detected that there is no element overflow at the edge and outline position of the first picture, determining that the pre-layout user interface is adapted to the device parameters;

[0013] If it is detected that there is element overflow at the edge and outline position in the first picture, it is determined that the pre-layout user interface is not suitable for the device parameters.

[0014] Optionally, determining, according to the display interface, whether the pre-layout user interface is adapted to the device parameters includes:

[0015] Acquire a first picture corresponding to the display interface and a second picture corresponding to the pre-layout user interface;

[0016] According to the first picture and the second picture, it is determined whether the pre-layout user interface is adapted to the device parameters.

[0017] Optionally, the acquiring a first picture corresponding to the display interface and a second picture corresponding to the pre-layout user interface includes:

[0018] Calling a screenshot instruction to take a screenshot of the display interface to obtain a first picture corresponding to the display interface;

[0019] A screenshot is taken of the pre-layout user interface to obtain a second picture corresponding to the pre-layout user interface.

[0020] Optionally, determining whether the pre-layout user interface is adapted to the device parameter according to the first picture and the second picture includes:

[0021] Determining a first element in the first picture and a second element in the second picture;

[0022] determining a first coordinate of the first element and a second coordinate of the second element;

[0023] According to the first coordinate and the second coordinate, it is determined whether the pre-layout user interface is adapted to the device parameter.

[0024] Optionally, determining whether the pre-layout user interface is adapted to the device parameter according to the first coordinate and the second coordinate includes:

[0025] Determining whether an error between the first coordinate and the second coordinate is within a preset error range;

[0026] If the error between the first coordinate and the second coordinate is within a preset error range, determining that the pre-layout user interface is adapted to the device parameter;

[0027] If the error between the first coordinate and the second coordinate is not within a preset error range, it is determined that the pre-layout user interface is not compatible with the device parameters.

[0028] Optionally, the determining the first element in the first picture and the second element in the second picture includes:

[0029] Preprocessing the first image, and extracting features from the preprocessed first image to obtain a first feature; the preprocessing includes: grayscale, denoising, and edge detection;

[0030] Determine a first element in the first picture according to the first feature;

[0031] Preprocessing the second image, and extracting features from the preprocessed second image to obtain second features;

[0032] A second element in the second picture is determined according to the second feature.

[0033] Optionally, after determining that the pre-layout user interface is not compatible with the device parameters, the method further includes:

[0034] Determining, from the multiple screen sizes, the multiple resolutions, and the multiple operating systems, a target screen size, a target resolution, and a target operating system for which the pre-layout user interface is not compatible;

[0035] The target screen size, the target resolution, and the target operating system are output.

[0036] Optionally, determining the first coordinate of the first element and the second coordinate of the second element includes:

[0037] The first coordinates of the first element and the second coordinates of the second element are determined by using projection positioning technology.

[0038] An embodiment of the present invention discloses a user interface testing device, the device comprising:

[0039] The first acquisition module is used to deploy the test environment and obtain automated test cases;

[0040] A second acquisition module is used to acquire device parameters of the simulated device and a pre-layout user interface; the device parameters include: multiple screen sizes, multiple resolutions and multiple operating systems;

[0041] An output module, used for executing the automated test case in the test environment, and outputting display interfaces of the pre-layout user interface under the multiple screen sizes, the multiple resolutions and the multiple operating systems;

[0042] The first determination module is used to determine whether the pre-layout user interface is adapted to the device parameters according to the display interface.

[0043] Optionally, the first determining module includes:

[0044] A calling submodule is used to call a screenshot instruction to take a screenshot of the display interface to obtain a first picture;

[0045] A detection submodule, configured to input the first image into a detection model to detect whether there is element overflow at edge and contour positions in the first image; the detection model is a model obtained by pre-training a neural network model;

[0046] A first determination submodule is configured to determine that the pre-layout user interface is adapted to the device parameters if it is detected that no element overflow exists at the edge and contour positions in the first picture;

[0047] The second determination submodule is configured to determine that the pre-layout user interface is not compatible with the device parameters if element overflow is detected at edge and outline positions in the first image.

[0048] Optionally, the first determining module includes:

[0049] An acquisition submodule, used to acquire a first picture corresponding to the display interface and a second picture corresponding to the pre-layout user interface;

[0050] The third determination submodule is used to determine whether the pre-layout user interface is adapted to the device parameters according to the first picture and the second picture.

[0051] Optionally, the acquisition submodule includes:

[0052] A calling unit, configured to call a screenshot instruction to take a screenshot of the display interface to obtain a first picture corresponding to the display interface;

[0053] The screenshot unit is used to take a screenshot of the pre-layout user interface to obtain a second picture corresponding to the pre-layout user interface.

[0054] Optionally, the third determining submodule includes:

[0055] A first determining unit, configured to determine a first element in the first picture and a second element in the second picture;

[0056] A second determining unit, used to determine a first coordinate of the first element and a second coordinate of the second element;

[0057] The third determining unit is used to determine whether the pre-layout user interface is adapted to the device parameters according to the first coordinates and the second coordinates.

[0058] Optionally, the third determining unit includes:

[0059] A judging subunit, used to judge whether the error between the first coordinate and the second coordinate is within a preset error range;

[0060] A first determining subunit, configured to determine that the pre-layout user interface is adapted to the device parameter if an error between the first coordinate and the second coordinate is within a preset error range;

[0061] The second determining subunit is configured to determine that the pre-layout user interface is not compatible with the device parameter if the error between the first coordinate and the second coordinate is not within a preset error range.

[0062] Optionally, the first determining unit includes:

[0063] A first extraction subunit is used to preprocess the first image and perform feature extraction on the preprocessed first image to obtain a first feature; the preprocessing includes: grayscale, denoising and edge detection;

[0064] A third determining subunit, configured to determine a first element in the first picture according to the first feature;

[0065] A second extraction subunit is used to preprocess the second image and perform feature extraction on the preprocessed second image to obtain a second feature;

[0066] The fourth determining subunit is used to determine a second element in the second picture according to the second feature.

[0067] Optionally, the device further comprises:

[0068] A second determining module is used to determine, from the multiple screen sizes, the multiple resolutions and the multiple operating systems, a target screen size, a target resolution and a target operating system that the pre-layout user interface is not compatible with;

[0069] An output module is used to output the target screen size, the target resolution and the target operating system.

[0070] Optionally, the second determining unit includes:

[0071] The fifth determining subunit is used to determine the first coordinates of the first element and the second coordinates of the second element by using a projection positioning technology.

[0072] The present invention also discloses an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned user interface testing method when executing the computer program.

[0073] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the user interface testing method described above are implemented.

[0074] The embodiments of the present invention include the following advantages:

[0075] In an embodiment of the present invention, a test environment is first deployed, and automated test cases are obtained, and then device parameters of a simulated device and a pre-layout user interface are obtained, wherein the device parameters include: multiple screen sizes, multiple resolutions, and multiple operating systems. Then, automated test cases are executed in the test environment, and display interfaces pre-layout in the user interface under multiple screen sizes, multiple resolutions, and multiple operating systems are output. Finally, based on the display interface, it is determined whether the pre-layout user interface is adapted to the device parameters, so that the device parameters of the simulated device are used to test whether the pre-layout user interface matches multiple screen sizes, multiple resolutions, and multiple operating systems. At the same time, there is no need to rely on manual testing. Through automated testing, labor costs are reduced, and test efficiency and accuracy are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 is a flowchart of the steps of a user interface testing method provided by an embodiment of the present invention;

[0077] Figure 2 is a structural block diagram of a user interface testing device provided by an embodiment of the present invention;

[0078] Figure 3 is a structural block diagram of an electronic device provided by an embodiment of the present invention;

[0079] Figure 4 It is a structural block diagram of a computer-readable storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0080] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0081] In the related art, the user interface testing method often needs to rely on manual operation testing and manual visual inspection, which is not only inefficient but also prone to errors. In order to solve the above technical problems, the present invention provides a user interface testing method in an embodiment of the present invention, which first deploys a test environment and obtains automated test cases, then obtains device parameters of a simulated device and a pre-layout user interface, wherein the device parameters include: multiple screen sizes, multiple resolutions, and multiple operating systems, and then executes automated test cases in the test environment to output a display interface pre-layout in the user interface under multiple screen sizes, multiple resolutions, and multiple operating systems, and finally determines whether the pre-layout user interface is adapted to the device parameters according to the display interface, thereby testing whether the pre-layout user interface matches multiple screen sizes, multiple resolutions, and multiple operating systems through the device parameters of the simulated device. At the same time, it does not need to rely on manual testing, and through automated testing, it reduces labor costs and improves test efficiency and accuracy.

[0082] Reference Figure 1 , shows a flowchart of a user interface testing method provided by an embodiment of the present invention, the method may specifically include the following steps:

[0083] Step 101: deploy a test environment and obtain automated test cases.

[0084] In an embodiment of the present invention, when testing the user interface, the test environment can be deployed first, and the automated testing tool Appium can be deployed. The user interface can be automatically tested using the automated testing tool Appium. After the test environment is deployed, the automated test cases can be obtained. The automated test cases in the present invention can be pre-written by the tester. When writing the automated test cases, the tester can use Python language to write them. It should be noted that other automated testing tools can also be used in the present invention. The present invention does not specifically limit the specific testing tools. The present invention can also use other languages ​​to write automated test cases in the present invention. The present invention does not specifically limit the specific writing language.

[0085] Appium, an automated testing tool, is an open source automated testing tool that is mainly used for automated testing of mobile applications. It supports multiple mobile operating systems, including iOS and Android. The design concept of Appium, an automated testing tool, is to provide a cross-platform, unified interface so that testers can use the same API to write test scripts for different platforms. Appium, an automated testing tool, allows the use of the same API to write test scripts for iOS and Android, which means there is no need to write different test code for different platforms. Appium, an automated testing tool, supports multiple programming languages, including Java, Python, Ruby, JavaScript (Node.js), C#, PHP, and Perl. Appium, an automated testing tool, uses the Selenium WebDriver protocol, which makes it compatible with the Selenium ecosystem and can take advantage of Selenium's tools and libraries. Appium, an automated testing tool, does not require any code or library to be embedded in the application, which means that any application can be automated, including third-party applications. Appium, an automated testing tool, can test native applications, hybrid applications (such as applications developed using Cordova or React Native), and mobile web applications.

[0086] Step 102, obtaining device parameters of the simulated device and a pre-layout user interface; the device parameters include: multiple screen sizes, multiple resolutions, and multiple operating systems.

[0087] In an embodiment of the present invention, after the test environment is deployed and the automated test cases are obtained, the device parameters of the simulated device and the pre-layout user interface can be obtained. The tester can adjust the device parameters of the simulated device based on the automated test tool Appium, where the device parameters may include: multiple screen sizes, multiple resolutions, and multiple operating systems. In the present invention, the BrowserStack tool can also be used to simulate the display effects on different devices by modifying the screen size and resolution of the simulated device. When obtaining the device parameters of the model device, the pre-layout user interface can also be obtained at the same time, and the pre-layout user interface is the user interface that needs to be tested.

[0088] The BrowserStack tool is a cloud-based cross-browser testing platform that allows developers and testers to test their websites and mobile applications on different browsers, operating systems, and real mobile devices. BrowserStack provides a range of testing tools and services to ensure that web applications run seamlessly in different environments.

[0089] The main features of the BrowserStack tool include: BrowserStack allows you to test your website on a wide range of browsers (including older versions) and operating systems, ensuring compatibility and performance in different environments. With live testing, you can interact with the website in real time on different devices and browsers. This helps to quickly identify and fix problems. BrowserStack integrates with popular automated testing frameworks such as Selenium, Appium, and Cypress. It allows automated tests to be run in parallel on a large number of browser and device combinations, significantly reducing the time required for testing. BrowserStack provides a large number of real mobile devices on which mobile applications can be tested to ensure that they work properly on different devices and operating systems. If the website or application needs to access local servers or intranet resources, BrowserStack's local testing feature allows testing through the BrowserStack Local agent. BrowserStack can be integrated with various development and testing tools, such as Jenkins, Travis CI, GitHub, etc., to provide a seamless continuous integration and continuous delivery (CI / CD) process.

[0090] Step 103 , executing the automated test case in the test environment, and outputting display interfaces of the pre-layout user interface under the multiple screen sizes, the multiple resolutions, and the multiple operating systems.

[0091] In an embodiment of the present invention, after obtaining the device parameters of the simulated device and the pre-layout user interface, the automated test case can be executed in the deployed test environment. It should be noted that when executing the automated test case, the present application can be executed based on multiple screen sizes, multiple resolutions and multiple operating systems. After executing the automated test case, the pre-layout user interface can be output as a display interface under multiple screen sizes, multiple resolutions and multiple operating systems. The display interface in the present invention is the interface displayed by the pre-layout user interface under multiple screen sizes, multiple resolutions and multiple operating systems.

[0092] Step 104: Determine, based on the display interface, whether the pre-layout user interface is adapted to the device parameters.

[0093] In an embodiment of the present invention, after outputting the display interfaces of the pre-layout user interface under multiple screen sizes, multiple resolutions and multiple operating systems respectively, it can be determined whether the pre-layout user interface matches the device parameters based on the display interfaces, so that it can be determined through automated testing whether the pre-layout user interface matches the multiple screen sizes, multiple resolutions and multiple operating systems, thereby improving the test efficiency, and corresponding tests can be performed for different screen sizes, resolutions and operating systems.

[0094] In an embodiment of the present invention, a test environment is deployed, and an automated test case is obtained; the device parameters of the simulated device and the pre-layout user interface are obtained; the device parameters include: multiple screen sizes, multiple resolutions, and multiple operating systems; the automated test case is executed in the test environment, and the display interface of the pre-layout user interface under multiple screen sizes, multiple resolutions, and multiple operating systems is output; according to the display interface, it is determined whether the pre-layout user interface is adapted to the device parameters. Thus, through the device parameters of the simulated device, it is tested whether the pre-layout user interface matches multiple screen sizes, multiple resolutions, and multiple operating systems. At the same time, there is no need to rely on manual testing. Through automated testing, the labor cost is reduced, and the test efficiency and accuracy are improved.

[0095] In one embodiment, determining whether the pre-layout user interface is adapted to the device parameters based on the display interface may include: calling a screenshot instruction to take a screenshot of the display interface to obtain a first image; inputting the first image into a detection model to detect whether there is element overflow at the edge and contour position in the first image; the detection model is a model obtained by a pre-trained neural network model; if it is detected that there is no element overflow at the edge and contour position in the first image, determining that the pre-layout user interface is adapted to the device parameters; if it is detected that there is element overflow at the edge and contour position in the first image, determining that the pre-layout user interface is not adapted to the device parameters.

[0096] Specifically, when determining whether the pre-layout user interface is adapted to the device parameters, the screenshot command in the test tool can be called, such as river.get_screenshot_as_file(filename) or driver.get_screenshot_as_png(). Through the screenshot command, the display interface can be screenshotted to obtain a first image corresponding to the display interface, and then the obtained first image is input into the detection model. The detection model is used to detect whether there is element overflow at the edge and contour position in the first image. If it is detected that there is no element overflow at the edge and contour position in the first image, it proves that the display interface is adapted to the device parameters, and further proves that the pre-layout user interface is adapted to the device parameters. If it is detected that there is element overflow at the edge and contour position in the first image, it proves that the display interface is not compatible with the device parameters, and further proves that the pre-layout user interface is not compatible with the device parameters. It should be noted that in the present invention, screenshots can also be taken through other screenshot commands. For specific screenshot commands, they can be set according to actual needs, and the present invention is not specifically limited here.

[0097] It should be noted that the detection model in the present invention can be a model obtained by pre-training a neural network model, and the training process is as follows: first collect sample data, then pre-process the sample data, select the neural network model to be trained, train the selected neural network model through the pre-processed sample data, obtain the trained neural network model, and determine the trained neural network model as the detection model.

[0098] In one embodiment, determining whether a pre-layout user interface is adapted to device parameters based on a display interface may include: acquiring a first image corresponding to the display interface and a second image corresponding to the pre-layout user interface; and determining whether the pre-layout user interface is adapted to device parameters based on the first image and the second image.

[0099] In one embodiment, obtaining a first image corresponding to a display interface and a second image corresponding to a pre-layout user interface may include: calling a screenshot instruction to take a screenshot of the display interface to obtain the first image corresponding to the display interface; taking a screenshot of the pre-layout user interface to obtain the second image corresponding to the pre-layout user interface.

[0100] Specifically, when it is necessary to obtain the first image and the second image, the screenshot command in the test tool is called, such as river.get_screenshot_as_file(filename) or driver.get_screenshot_as_png(). Through the screenshot command, the display interface can be screenshotted to obtain the first image corresponding to the display interface, and the pre-layout user interface can be screenshotted to obtain the second image corresponding to the pre-layout user interface. It should be noted that other screenshot commands can also be used in the present invention to take screenshots. The specific screenshot command can be set according to actual needs, and the present invention does not make specific limitations here.

[0101] In one embodiment, determining whether a pre-layout user interface is adapted to the device parameters based on a first picture and a second picture may include: determining a first element in the first picture and a second element in the second picture; determining a first coordinate of the first element and a second coordinate of the second element; and determining whether the pre-layout user interface is adapted to the device parameters based on the first coordinate and the second coordinate.

[0102] Specifically, the first picture corresponding to the display interface may include a first element, and the second picture corresponding to the pre-layout user interface may include a second element. The corresponding first element is first determined from the first picture, and the corresponding second element is determined from the second picture. Then, the first coordinate of the first element and the second coordinate of the second element are determined. The first coordinate is compared with the second coordinate to determine whether the display interface and the pre-layout user interface are the same or similar, so as to determine whether the pre-layout user interface is adapted to the device parameters, that is, the display interface is compared with the pre-layout user interface to determine whether the pre-layout user interface is adapted to the device parameters.

[0103] In one embodiment, determining a first element in a first image and a second element in a second image includes: preprocessing the first image and performing feature extraction on the preprocessed first image to obtain a first feature; the preprocessing includes: graying, denoising and edge detection; determining the first element in the first image based on the first feature; preprocessing the second image and performing feature extraction on the preprocessed second image to obtain a second feature; and determining the second element in the second image based on the second feature.

[0104] Specifically, when determining the first element in the first image and the second element in the second image, the first image may be preprocessed by grayscale, denoising and edge detection, and feature extraction may be performed on the preprocessed first image, for example, extracting key points and descriptors in the first image, and determining the corresponding first element from the first image based on the extracted key points and descriptors, and preprocessing the second image by grayscale, denoising and edge detection, and feature extraction may be performed on the preprocessed second image, for example, extracting key points and descriptors in the second image, and determining the corresponding second element from the second image based on the extracted key points and descriptors. The first feature and the second feature in the present invention may also be extracted in other ways, and the present invention does not specifically limit the specific extraction method.

[0105] In one embodiment, determining the first coordinate of the first element and the second coordinate of the second element may include: determining the first coordinate of the first element and the second coordinate of the second element by using a projection positioning technology.

[0106] Specifically, the projection of the first image in the horizontal direction can be calculated, that is, the number of non-zero pixels in each row can be counted, the projection of the first image in the vertical direction can be calculated, that is, the number of non-zero pixels in each column can be counted, and the first coordinate of the first element in the first image can be determined according to the projection result. The projection of the second image in the horizontal direction can be calculated, that is, the number of non-zero pixels in each row can be counted, and the projection of the second image in the vertical direction can be calculated, that is, the number of non-zero pixels in each column can be counted, and the second coordinate of the second element in the second image can be determined according to the projection result.

[0107] In one embodiment, determining whether a pre-layout user interface is adapted to device parameters based on the first coordinate and the second coordinate may include: judging whether an error between the first coordinate and the second coordinate is within a preset error range; if the error between the first coordinate and the second coordinate is within the preset error range, determining that the pre-layout user interface is adapted to the device parameters; if the error between the first coordinate and the second coordinate is not within the preset error range, determining that the pre-layout user interface is not adapted to the device parameters.

[0108] Specifically, after determining the first coordinate of the first element in the first image and the second coordinate of the second element in the second image, it can be determined whether the error between the first coordinate and the second coordinate is within a preset error range, where the preset range can be set according to actual needs, and the present invention is not specifically limited here. For example, if it is necessary to ensure that the first coordinate and the second coordinate are the same, the preset error range can be set to zero, that is, the position of the first element in the first image is the same as the position of the second element in the second image.

[0109] If the error between the first coordinate and the second coordinate is within the preset error range, it can be determined that the position of the first element in the first picture is almost the same as the position of the second element in the second picture, which also indicates that the first picture is almost the same as the second picture, and further indicates that the display interface and the pre-layout user interface are also almost the same, thus proving that the pre-layout user interface adapts to the device parameters.

[0110] If the error between the first coordinate and the second coordinate is not within the preset error range, it can be determined that the position of the first element in the first picture is different from the position of the second element in the second picture, which also indicates that the first picture is different from the second picture, and further indicates that the display interface and the pre-layout user interface are also different, then it is proved whether the pre-layout user interface is suitable for the device parameters.

[0111] In an embodiment of the present invention, after determining that the pre-layout user interface is not compatible with device parameters, the method may also include: determining a target screen size, target resolution, and target operating system that the pre-layout user interface is not compatible with from a plurality of screen sizes, a plurality of resolutions, and a plurality of operating systems; and outputting the target screen size, target resolution, and target operating system.

[0112] For example: the multiple screen sizes in the present invention are the first size, the second size, and the third size, the multiple resolutions are the first resolution and the second resolution, and the multiple operating systems are the first operating system and the second operating system. When it is detected that the pre-layout user interface does not match the device parameters, that is, when it is detected that the display interface does not match the device parameters, the device parameters that the display interface does not match are determined, such as: the incompatible parameters are the second size, the first resolution, and the second operating system. At this time, the determined second size, first resolution, and second operating system can be determined as target parameters that the pre-layout user interface does not match, and the target parameters of the second size, first resolution, and second operating system are output, so that the tester can clearly know which target parameters the pre-layout user interface does not match, and the tester can make corresponding adjustments according to the target parameters.

[0113] In the embodiment of the present invention, when the user interface is automatically tested, a corresponding test log can be generated and saved to a specified location so that the tester can view the test log at any time. At the same time, the test results can also be output and saved.

[0114] In an embodiment of the present invention, a test environment is deployed, and an automated test case is obtained; the device parameters of the simulated device and the pre-layout user interface are obtained; the device parameters include: multiple screen sizes, multiple resolutions, and multiple operating systems; the automated test case is executed in the test environment, and the display interface of the pre-layout user interface under multiple screen sizes, multiple resolutions, and multiple operating systems is output; according to the display interface, it is determined whether the pre-layout user interface is adapted to the device parameters. Thus, through the device parameters of the simulated device, it is tested whether the pre-layout user interface matches multiple screen sizes, multiple resolutions, and multiple operating systems. At the same time, there is no need to rely on manual testing. Through automated testing, the labor cost is reduced, and the test efficiency and accuracy are improved.

[0115] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0116] Reference Figure 2 , shows a structural block diagram of a user interface testing device provided by an embodiment of the present invention, which may specifically include the following modules:

[0117] The first acquisition module 201 is used to deploy a test environment and acquire automated test cases;

[0118] The second acquisition module 202 is used to acquire device parameters of the simulated device and a pre-layout user interface; the device parameters include: multiple screen sizes, multiple resolutions and multiple operating systems;

[0119] An output module 203 is used to execute the automated test case in the test environment and output the display interfaces of the pre-layout user interface under the multiple screen sizes, the multiple resolutions and the multiple operating systems;

[0120] The first determination module 204 is used to determine whether the pre-layout user interface is adapted to the device parameters according to the display interface.

[0121] In one embodiment, the first determining module 204 includes:

[0122] A calling submodule is used to call a screenshot instruction to take a screenshot of the display interface to obtain a first picture;

[0123] A detection submodule, configured to input the first image into a detection model to detect whether there is element overflow at edge and contour positions in the first image; the detection model is a model obtained by pre-training a neural network model;

[0124] A first determination submodule is configured to determine that the pre-layout user interface is adapted to the device parameters if it is detected that no element overflow exists at the edge and contour positions in the first picture;

[0125] The second determination submodule is configured to determine that the pre-layout user interface is not compatible with the device parameters if element overflow is detected at edge and outline positions in the first image.

[0126] In one embodiment, the first determining module 204 includes:

[0127] An acquisition submodule, used to acquire a first picture corresponding to the display interface and a second picture corresponding to the pre-layout user interface;

[0128] The third determination submodule is used to determine whether the pre-layout user interface is adapted to the device parameters according to the first picture and the second picture.

[0129] In one embodiment, the acquisition submodule includes:

[0130] A calling unit, configured to call a screenshot instruction to take a screenshot of the display interface to obtain a first picture corresponding to the display interface;

[0131] The screenshot unit is used to take a screenshot of the pre-layout user interface to obtain a second picture corresponding to the pre-layout user interface.

[0132] In one embodiment, the third determining submodule includes:

[0133] A first determining unit, configured to determine a first element in the first picture and a second element in the second picture;

[0134] A second determining unit, used to determine a first coordinate of the first element and a second coordinate of the second element;

[0135] The third determining unit is used to determine whether the pre-layout user interface is adapted to the device parameters according to the first coordinates and the second coordinates.

[0136] In one embodiment, the third determining unit includes:

[0137] A judging subunit, used to judge whether the error between the first coordinate and the second coordinate is within a preset error range;

[0138] A first determining subunit, configured to determine that the pre-layout user interface is adapted to the device parameter if an error between the first coordinate and the second coordinate is within a preset error range;

[0139] The second determining subunit is configured to determine that the pre-layout user interface is not compatible with the device parameter if the error between the first coordinate and the second coordinate is not within a preset error range.

[0140] In one embodiment, the first determining unit includes:

[0141] A first extraction subunit is used to preprocess the first image and perform feature extraction on the preprocessed first image to obtain a first feature; the preprocessing includes: grayscale, denoising and edge detection;

[0142] A third determining subunit, configured to determine a first element in the first picture according to the first feature;

[0143] A second extraction subunit is used to preprocess the second image and perform feature extraction on the preprocessed second image to obtain a second feature;

[0144] The fourth determining subunit is used to determine a second element in the second picture according to the second feature.

[0145] In one embodiment, the device further comprises:

[0146] A second determining module is used to determine, from the multiple screen sizes, the multiple resolutions and the multiple operating systems, a target screen size, a target resolution and a target operating system that the pre-layout user interface is not compatible with;

[0147] An output module is used to output the target screen size, the target resolution and the target operating system.

[0148] In one embodiment, the second determining unit includes:

[0149] The fifth determining subunit is used to determine the first coordinates of the first element and the second coordinates of the second element by using a projection positioning technology.

[0150] In the embodiment of the present invention, the first acquisition module is used to deploy the test environment and obtain the automated test cases; the second acquisition module is used to obtain the device parameters of the simulated device and the pre-layout user interface; the device parameters include: multiple screen sizes, multiple resolutions and multiple operating systems; the output module is used to execute the automated test cases in the test environment and output the display interface of the pre-layout user interface under multiple screen sizes, multiple resolutions and multiple operating systems; the first determination module is used to determine whether the pre-layout user interface is adapted to the device parameters according to the display interface. Thus, through the device parameters of the simulated device, it is tested whether the pre-layout user interface matches multiple screen sizes, multiple resolutions and multiple operating systems. At the same time, there is no need to rely on manual testing. Through automated testing, the labor cost is reduced and the test efficiency and accuracy are improved.

[0151] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0152] like Figure 3 As shown, an embodiment of the present invention provides a structural block diagram of an electronic device, including:

[0153] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned user interface testing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0154] like Figure 4 As shown, an embodiment of the present invention provides a structural block diagram of a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned user interface testing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0155] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0156] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0157] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0160] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0161] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0162] The user interface testing method, user interface testing device, electronic device and computer-readable storage medium provided by the present invention are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A user interface testing method, characterized in that: The method comprises: Deploy the test environment and obtain automated test cases; Obtaining device parameters of a simulated device and a pre-layout user interface; the device parameters include: multiple screen sizes, multiple resolutions, and multiple operating systems; Execute the automated test case in the test environment, and output display interfaces of the pre-layout user interface under the multiple screen sizes, the multiple resolutions, and the multiple operating systems; According to the display interface, it is determined whether the pre-layout user interface is adapted to the device parameters.

2. The user interface testing method according to claim 1, characterized in that: The determining, according to the display interface, whether the pre-layout user interface is adapted to the device parameters includes: Calling a screenshot command to take a screenshot of the display interface to obtain a first picture; Inputting the first image into a detection model to detect whether there is element overflow at edge and contour positions in the first image; the detection model is a model obtained by pre-training a neural network model; If it is detected that there is no element overflow at the edge and outline position of the first picture, determining that the pre-layout user interface is adapted to the device parameters; If it is detected that there is element overflow at the edge and outline position in the first picture, it is determined that the pre-layout user interface is not suitable for the device parameters.

3. The user interface testing method according to claim 1, characterized in that: The determining, according to the display interface, whether the pre-layout user interface is adapted to the device parameters includes: Acquire a first picture corresponding to the display interface and a second picture corresponding to the pre-layout user interface; According to the first picture and the second picture, it is determined whether the pre-layout user interface is adapted to the device parameters.

4. The user interface testing method according to claim 3, characterized in that: The acquiring of a first picture corresponding to the display interface and a second picture corresponding to the pre-layout user interface includes: Calling a screenshot instruction to take a screenshot of the display interface to obtain a first picture corresponding to the display interface; A screenshot is taken of the pre-layout user interface to obtain a second picture corresponding to the pre-layout user interface.

5. The user interface testing method according to claim 3, characterized in that: The determining, according to the first picture and the second picture, whether the pre-layout user interface is adapted to the device parameter includes: Determining a first element in the first picture and a second element in the second picture; determining a first coordinate of the first element and a second coordinate of the second element; According to the first coordinate and the second coordinate, it is determined whether the pre-layout user interface is adapted to the device parameter.

6. The user interface testing method according to claim 5, characterized in that: The determining, according to the first coordinate and the second coordinate, whether the pre-layout user interface is adapted to the device parameter includes: Determining whether an error between the first coordinate and the second coordinate is within a preset error range; If the error between the first coordinate and the second coordinate is within a preset error range, determining that the pre-layout user interface is adapted to the device parameter; If the error between the first coordinate and the second coordinate is not within a preset error range, it is determined that the pre-layout user interface is not compatible with the device parameters.

7. The user interface testing method according to claim 5, characterized in that: The determining a first element in the first picture and a second element in the second picture includes: Preprocessing the first image, and extracting features from the preprocessed first image to obtain a first feature; the preprocessing includes: grayscale, denoising, and edge detection; Determine a first element in the first picture according to the first feature; Preprocessing the second image, and extracting features from the preprocessed second image to obtain second features; A second element in the second picture is determined according to the second feature.

8. The user interface testing method according to claim 2 or 6, characterized in that: After determining that the pre-layout user interface is not compatible with the device parameters, the method further includes: Determining, from the multiple screen sizes, the multiple resolutions, and the multiple operating systems, a target screen size, a target resolution, and a target operating system for which the pre-layout user interface is not compatible; The target screen size, the target resolution, and the target operating system are output.

9. The user interface testing method according to claim 5, characterized in that: The determining the first coordinate of the first element and the second coordinate of the second element comprises: The first coordinates of the first element and the second coordinates of the second element are determined by using projection positioning technology.

10. A user interface testing device, characterized in that: The device comprises: The first acquisition module is used to deploy the test environment and obtain automated test cases; A second acquisition module is used to acquire device parameters of the simulated device and a pre-layout user interface; the device parameters include: multiple screen sizes, multiple resolutions and multiple operating systems; An output module, used for executing the automated test case in the test environment, and outputting display interfaces of the pre-layout user interface under the multiple screen sizes, the multiple resolutions and the multiple operating systems; The first determination module is used to determine whether the pre-layout user interface is adapted to the device parameters according to the display interface.

11. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the user interface testing method according to any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the user interface testing method according to any one of claims 1 to 9 are implemented.