First device, second device, testing method, system, and medium

Abstract

A first device, a second device, a testing method, a system, and a medium, relating to the technical field of testing. The first device comprises: a processor, wherein a program runs in the processor, and during running of the program in the processor, a target object on the first device or the second device is automatically tested. The automatic test comprises: acquiring test information sent to the first device, the test information being used for describing pending test content for the target object; analyzing the test information to obtain a plurality of key texts related to the test, and constructing the plurality of key texts into an action sequence of a target data structure, the action sequence being used for representing an operation procedure for the target object during execution of the test content; and mapping the action sequence to an operation command adapted to the target object, and executing the operation command on the target object, so as to complete the test indicated by the test information for the target object.

Description

First device, second device, testing method, system, and medium Technical Field

[0001] The present disclosure relates to the technical field of software and hardware testing, and in particular to a first device, a second device, a testing method, a system, and a medium. Background Art

[0002] With the development of electronic information technology, various electronic devices and terminal applications are emerging in an endless stream. Among them, electronic devices and terminal applications generally go through three important stages: development, testing, and delivery. These three stages, especially the testing stage, play an important role in providing users with electronic devices and terminal applications with stable performance.

[0003] Overview

[0004] A first device, a second device, a testing method, a system, and a medium.

[0005] A first device includes a processor, wherein the processor runs a program, and when the program is run by the processor, the program automatically tests a target object on the first device or the second device; wherein the automatic test includes:

[0006] Acquire test information sent to the first device, where the test information is used to describe content of a test to be performed on the target object;

[0007] Parsing multiple key texts related to the test from the test information, and constructing the multiple key texts into an action sequence of a target data structure, wherein the action sequence is used to represent an operation flow performed on the target object when executing the test content;

[0008] The action sequence is mapped into an operation command adapted to the target object, and the operation command is executed on the target object to complete the test indicated by the test information on the target object.

[0009] Exemplarily, the first device further includes a memory, wherein the memory stores a plurality of library files associated with the program, wherein different types of library files correspond to key texts with different attributes in the action sequence, and the step of mapping the action sequence into an operation command adapted to the target object includes:

[0010] For each key text in the action sequence, the key text is mapped to a library file corresponding to the attribute of the key text to obtain a command line corresponding to the key text; wherein the library file includes multiple preset command lines corresponding to multiple keywords under the same attribute;

[0011] The operation command is generated based on command lines corresponding to the multiple key texts.

[0012] Exemplarily, the attributes include action attributes, element attributes, and object attributes, and the key text of the object attributes is used to identify the target object; wherein the multiple library files include an action library file corresponding to the action attributes, a test database file corresponding to the element attributes, and an object library file corresponding to the object attributes;

[0013] The action library file includes preset command lines corresponding to various operation actions, the test database file includes preset command lines corresponding to different input parameters, and the object library file includes preset command lines for locating different objects.

[0014] Exemplarily, the step of parsing a plurality of key texts related to the test from the test information includes:

[0015] Segmenting the test information to obtain multiple paragraphs;

[0016] Based on a preset blacklist and / or whitelist, the key text is extracted from each of the paragraphs.

[0017] Exemplarily, the step of calling the interface where the target object is located and executing the operation command on the target object includes:

[0018] In a case where the target object includes an object in a web page, calling an interface of the web page to simulate an input device of a device where the target object is located to perform an operation corresponding to the operation command on the target object;

[0019] In the case where the target object includes a writing page, the interface of the writing page is called to perform virtual writing on the writing page according to the operation command line, or the interface of the mechanical device is called to perform physical writing on the writing page according to the operation command line.

[0020] Exemplarily, the target object includes an object in a web page, the test information further includes target identification information describing the target object, and before the step of executing the operation command on the target object, the processor is further configured to locate the target object when running the program; locating the target object includes:

[0021] Get the identification information of each object in the currently displayed web page;

[0022] The target identification information is matched with the plurality of identification information respectively to obtain the position of the target object in the currently displayed web page.

[0023] Exemplarily, the target identification information includes an appearance image and / or name of the target object, and the step of calling the device where the target object is located to obtain identification information of each object in the currently displayed web page includes:

[0024] Taking a screenshot of the currently displayed web page, and dividing the screenshot image into a plurality of image regions; wherein different objects correspond to different image regions;

[0025] Perform at least one of the following processing on the image area to obtain identification information corresponding to each of the objects:

[0026] Performing text recognition on the image area to obtain identification information of the text type of the object to match it with the name;

[0027] Feature extraction is performed on the image region to obtain identification information of the feature representation type to match it with the feature map of the appearance image.

[0028] Exemplarily, the first device further includes a display module, the target object includes the writing page, the test information includes first test information for performing a writing test, and the step of parsing a plurality of key texts related to the test from the test information includes:

[0029] In response to the first test information, controlling the display module to display a preset writing setting page; the writing setting page is pre-set with multiple writing templates, each of which includes at least one of the following setting items: line, line width, color, font, and background;

[0030] generating writing data in response to a selected writing template; and generating track point data in response to setting information received in the selected writing template;

[0031] determining the writing data and the track point data as the key text;

[0032] The step of executing the operation command on the target object includes:

[0033] Sending the writing data to the writing page, and calling a display component of the device where the writing page is located, so that the writing page responds to the received physical writing signal or virtual writing signal and generates a writing track on the writing page;

[0034] Among them, the physical writing signal is a touch signal triggered on the writing page by a mechanical device connected to the first device based on the track point data; the virtual writing signal is a virtual touch signal sent to the writing page by the display component based on the track point data.

[0035] Exemplarily, the test information includes assertion information, and the assertion information is used to indicate an expected test result of the target object; when the program is executed by the processor, it is further used to perform the following steps:

[0036] detecting a test process of the target object, obtaining process information, and sending the process information to a communication terminal communicating with the first device;

[0037] Obtaining the test results of the target object after the test is completed;

[0038] The test result and the assertion information are matched, and the matching result is output to a display component of the first device for display, and / or the matching result is sent to the communication terminal.

[0039] Exemplarily, the target object includes an object in a web page, and at least one of the following is detected for the target object during the test process: script data of the current web page, the display interface of the current web page, the frame rate of the current web page, the status information of the current web page, and the operation log of the current web page.

[0040] Exemplarily, when detecting the test process of the target object, the program, when executed by the processor, is also used to detect at least one of the following performances of the first device: the processor operating efficiency of the device where the target object is located, memory usage, data traffic of the current web page, power consumption of the device where the target object is located, usage rate of the graphics processor of the device where the target object is located, and network connection status of the device where the target object is located.

[0041] Exemplarily, the target object includes a display component, the test information includes first test information of a writing type, and the first test information is used to indicate that a writing test is performed on a writing page; and the step of obtaining a test result of the target object after the test is completed includes:

[0042] Calling an interface of a device where the writing page is located to obtain a display interface image after the writing test is completed, wherein the display interface image includes a writing track;

[0043] Recognize at least one of the following contents of the writing trace: font, line width, color, graphic shape, and pen shape;

[0044] The test result is obtained based on the recognition result and the assertion information.

[0045] Exemplarily, the memory on the first device further includes a use case database, the use case database including multiple test types and multiple preset test parameters corresponding to each test type, wherein different preset test parameters correspond to different test items under the same test type; when the program is executed by the processor, it is further configured to implement the following steps:

[0046] Based on the target test type carried in the test information, obtaining a plurality of target preset test parameters matching the target test type from the use case database;

[0047] Based on each of the target preset test parameters, a plurality of basic test information associated with the test information is generated; wherein different basic test information is used to implement different test items under the test information;

[0048] Sending a plurality of the basic test information to a use case source connected to the first device.

[0049] Exemplarily, the test information includes a target field, and the target field is used to indicate the second test information on which the current test information depends; when the program is executed by the processor, it further implements the following steps:

[0050] Responsive to the content of the target field, obtaining the second test information from a use case source in communication with the first apparatus;

[0051] Perform the test indicated by the second test information on the target object, and after the test is completed, execute the operation command on the target object to complete the test indicated by the test information on the target object.

[0052] There may also be a testing method, wherein the method comprises:

[0053] Acquire test information to be executed; wherein the test information is used to describe the test items to be performed on the target object;

[0054] Parsing multiple key texts related to the test from the test information, and constructing the multiple key texts into an action sequence of a target data structure, wherein the action sequence is used to represent an operation flow performed on the target object when executing the test content;

[0055] The action sequence is mapped into an operation command adapted to the target object, and the interface where the target object is located is called to execute the operation command on the target object, so as to complete the test indicated by the test information on the target object.

[0056] Exemplarily, mapping the action sequence into an operation command adapted to the target object includes:

[0057] For each key text in the action sequence, the key text is mapped to a library file corresponding to the attribute of the key text to obtain a command line corresponding to the key text; wherein the library file includes multiple preset command lines corresponding to multiple keywords under the same attribute, and different types of library files correspond to key texts with different attributes in the action sequence;

[0058] The operation command is generated based on command lines corresponding to the multiple key texts.

[0059] Exemplarily, the step of calling the interface where the target object is located and executing the operation command on the target object includes:

[0060] In a case where the target object includes an object in a web page, calling an interface of the web page to simulate an input device of a device where the target object is located to perform an operation corresponding to the operation command on the target object;

[0061] In the case where the target object includes a writing page, the interface of the writing page is called to execute virtual writing on the writing page according to the operation command line, or the interface of the mechanical device is called to execute physical writing on the writing page according to the operation command line.

[0062] Exemplarily, the target object includes an object in a web page, and the test information further includes target identification information describing the target object; before the step of calling the interface where the target object is located and executing the operation command on the target object, the method further includes:

[0063] Invoking the device where the target object is located to obtain identification information of each object in the currently displayed web page; wherein the identification information includes image information and / or name information of the object, and the target identification information includes appearance image and / or name information of the target object;

[0064] The target identification information is matched with the plurality of identification information respectively to obtain the position of the target object in the currently displayed web page.

[0065] Exemplarily, the target object includes a display component, the test information includes first test information of a written test type, and parsing a plurality of key texts related to the test from the test information includes:

[0066] In response to the first test information, a preset writing setting page is displayed; the writing setting page has a plurality of writing templates preset therein, each of the writing templates including at least one of the following setting items: line, line width, color, font, and background;

[0067] generating writing data in response to a selected writing template; and generating track point data in response to setting information received in the selected writing template;

[0068] A writing trajectory is generated on the writing page based on the trajectory point data and the writing data.

[0069] Exemplarily, the test information includes assertion information, where the assertion information is used to indicate an expected test result of the target object; the method further includes:

[0070] Detecting a test process of the target object, obtaining process information, and sending the process information to a communication terminal;

[0071] Calling the interface of the target object to obtain the test result of the target object after the test is completed;

[0072] The test result and the assertion information are matched, and the matching result is displayed and / or sent to the communication terminal.

[0073] Exemplarily, the test information includes first test information of a writing type, where the first test information is used to instruct to perform a writing test on a writing page; and calling an interface of the target object to obtain a test result of the target object after the test is completed includes:

[0074] Obtaining a display interface image of the writing page after the writing test is completed, wherein the display interface image includes a writing track;

[0075] Recognize at least one of the following contents of the writing trace: font, line width, color, graphic shape, and pen shape;

[0076] The test result is obtained based on the recognition result and the assertion information.

[0077] Exemplarily, the method further comprises:

[0078] Based on the target test type carried in the test information, obtain multiple target preset test parameters that match the target test type from a use case database; wherein the use case database includes multiple test types and multiple preset test parameters corresponding to each test type, wherein different preset test parameters correspond to different test items under the same test type;

[0079] Based on each of the target preset test parameters, a plurality of basic test information associated with the test information is generated; wherein different basic test information is used to implement different test items under the test information;

[0080] In response to calling any one of the plurality of basic test information, an automatic test indicated by the called basic test information is performed on the object to be tested.

[0081] A second device is also provided. The second device is configured with a target object and a display component, and a program that supports the operation of the target object. When the program is executed by a processor of the second device, it is configured to perform an automatic test instructed by a first device connected to the second device, wherein the automatic test includes:

[0082] receiving an operation command sent by the first device through an interface;

[0083] The simulated input device performs an operation corresponding to the operation command on the target object;

[0084] Displaying a picture after executing the operation command on the display component;

[0085] The operation command is obtained by mapping the standardized action sequence of the test information to the action sequence when the processor in the first device runs the program. The action sequence is used to represent the operation process performed on the target object when executing the test content.

[0086] Exemplarily, the operation command includes track point data and writing data, the display component is configured with a touch screen, the target object includes a writing page, and performing an operation corresponding to the operation command on the target object includes:

[0087] Based on the track point data, simulating a user's finger performing a virtual touch on the touch screen, and based on the writing data, generating a writing track corresponding to the position of the virtual touch on the writing page;

[0088] Alternatively, based on the operation command, in response to a touch on the touch screen by a mechanical device connected to the first device, a writing track corresponding to the touch position is generated on the writing page based on the writing data;

[0089] The mechanical device is used to perform touch control on the touch display screen based on the track point data sent by the first device.

[0090] Exemplarily, the second apparatus is configured with an operating device, and performing an operation corresponding to the operating command on the target object includes at least one of the following:

[0091] Simulating the operation device continuously touching the switch of a page button on the second device to change one of the following of the displayed page: display brightness, page open state, and page size;

[0092] simulating the operating device continuously touching the communication button on the second device to change the network connection state of the second device;

[0093] The operation device is simulated to switch the signal source connected to the second apparatus so as to jump from one signal source to another signal source; the signal source is used to provide a display page.

[0094] The present disclosure also provides a testing system, comprising a first device and a second device, wherein the first device is configured with a first page, and the second device is configured with a target object and a second page; wherein,

[0095] The first device is configured to, in response to an automatic test triggered on the first page, obtain test information, convert the test information into an action sequence, and map the action sequence into an operation command adapted to the target object; wherein the test information is used to describe the test items to be performed on the target object, and the action sequence is used to represent the operation process performed on the target object when executing the test content;

[0096] The second device is used to execute the operation command on the target object in response to the first device calling the interface where the target object is located, and display the execution process and execution result of the operation command on the second page.

[0097] A non-transitory computer-readable recording medium is also provided. The computer-readable recording medium records a program. When the program is executed by a processor in a first device, the program automatically tests a target object on the first device or the second device. The automatic testing includes:

[0098] Acquire test information sent to the first device; wherein the test information uses a custom format to describe the test items to be performed on the target object;

[0099] Converting the test information into a standardized action sequence, wherein the action sequence includes at least action parameters, object parameters, and configuration parameters, wherein the configuration parameters are used to indicate input configurations for the target object, the action parameters are used to indicate operations performed after the input, and the object parameters are used to describe the target object;

[0100] The action sequence is mapped into an operation command adapted to the target object, and the interface where the target object is located is called to execute the operation command on the target object, so as to complete the test indicated by the test information on the target object.

[0101] A first device is used, including a processor, which runs a program. When the program is executed by the processor, the program automatically tests a target object on the first device or the second device. The automatic test includes: obtaining test information sent to the first device, parsing multiple key texts related to the test from the test information, converting the multiple key texts into an action sequence of a target data structure, then mapping the action sequence into an operation command adapted to the target object, calling the interface of the target object, executing the operation command on the target object, and completing the test indicated by the test information on the target object. The action sequence is used to represent the operation process performed on the target object when executing the test content.

[0102] Because in the automatic test, the processor parses the key text from the test information sent to the first device, and then converts the key text into an operation command for operating the target object, so as to automatically complete the test operation on the target object; wherein, the analysis of the key text in the test information is automatically converted into an operation command for the target object based on the analysis and mapping. Therefore, when the first device is used to perform automatic testing, it is allowed to input only customized test information into the first device without writing a test script, thereby reducing the requirements for testers; at the same time, compared with the tester writing the script and then inputting it into the first device, the efficiency of automated parsing and mapping is greatly improved, which in turn improves the test efficiency of the target object. Therefore, the inspection efficiency before the target object is delivered can be improved, and the manpower and time costs required for inspection can be greatly shortened.

[0103] An embodiment of the present disclosure further discloses an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the test method when executed.

[0104] An embodiment of the present disclosure further discloses a computer-readable storage medium, which stores a computer program that enables a processor to execute the testing method described in the present disclosure.

[0105] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below.

[0106] BRIEF DESCRIPTION OF THE DRAWINGS

[0107] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following is a brief introduction to the drawings required for the description of the embodiments or related technologies. Obviously, the drawings described below are some embodiments of the present disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts. It should be noted that the scales in the drawings are for illustration only and do not represent the actual scale.

[0108] FIG1 is a schematic diagram showing a user scenario in which the embodiments of the present disclosure can be applied;

[0109] FIG2 shows an environmental architecture diagram of a first device in an automatic testing environment according to an embodiment of the present disclosure;

[0110] FIG3 shows a schematic diagram of the steps of the automatic test in an embodiment of the present disclosure;

[0111] FIG4 shows an operational architecture diagram of a first device for automatic testing in an embodiment of the present disclosure;

[0112] FIG5 is a schematic diagram showing a process of mapping an action sequence in an embodiment of the present disclosure;

[0113] FIG6 shows a schematic diagram of a scenario in which the writing function of a writing page is automatically tested in an embodiment of the present disclosure;

[0114] FIG7 shows a schematic diagram of a page template in a writing setting page according to an embodiment of the present disclosure;

[0115] FIG8 is a schematic diagram showing a flow chart of steps for monitoring a test process in an embodiment of the present disclosure;

[0116] FIG9 is a schematic diagram showing analysis of process information and test results according to an embodiment of the present disclosure;

[0117] FIG10 shows a schematic diagram of a communication environment architecture in which a first device is located according to an embodiment of the present disclosure;

[0118] FIG11a shows a flowchart of an embodiment of the present disclosure, taking the target object as a login control in a web page as an example;

[0119] FIG11b shows another flowchart of an embodiment of the present disclosure, taking the target object as a login control in a web page as an example;

[0120] FIG12 is a schematic diagram showing an environment architecture where the first device is located in an embodiment of the present disclosure;

[0121] FIG13 shows a flowchart of a writing test in an embodiment of the present disclosure;

[0122] FIG14 is a schematic diagram showing a flow chart of steps executed by the second device in an automatic test according to an embodiment of the present disclosure;

[0123] FIG15 shows a schematic diagram of a user scenario of the test system according to an embodiment of the present disclosure;

[0124] FIG16 is a schematic diagram showing a process flow of an automatic test performed by a test system according to an embodiment of the present disclosure;

[0125] FIG17 shows a schematic flow chart of the steps of the testing method in an embodiment of the present disclosure.

[0126] Detailed description

[0127] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0128] In the related art, during the inspection stage of electronic equipment and terminal applications, it is generally necessary to test the electrical stability of the electronic equipment and the operational stability of the terminal application. Among them, the test of the electrical stability of the electronic equipment includes the test of the stability of the electronic equipment in responding to external input signals, while the test of the operational stability of the terminal application will vary based on different terminal applications.

[0129] For example, in testing the stability of an electronic device's response to an external input signal, the response stability of the electronic device under frequent operation of an external infrared remote control signal will be tested; for another example, the power supply of the electronic device will be frequently cut off to determine the electrical stability of the electronic device; for another example, the network connection of the electronic device will be frequently cut off to test the network response stability of the electronic device.

[0130] Exemplarily, in the operation stability test of the terminal application, it may include login stability test, login account verification stability test, page display stability test, page jump stability test, comment publishing management stability test, etc.

[0131] The aforementioned tests generally require specialized testers. In related technologies, testers first compile a test script, then connect the test script to the interfaces of the electronic device and the terminal application. After the connection is complete, the test script is run and the test is performed. However, in a wide range of testing scenarios, the test objects and test purposes are diverse. Therefore, testers need to compile test scripts for each test object and purpose, which undoubtedly increases the difficulty and test time of the work, and also places higher demands on the professional level of the testers.

[0132] In view of this, the purpose of the present invention is to simplify the testing work of testers and avoid testers compiling test scripts as much as possible. A fully automatic testing process is proposed. In this testing process, the tester provides test information, which can be customized by the tester or filled in according to a standard format. The test information is submitted to a device. Next, the device parses and maps the test information into operation commands adapted to the target object, so that the customized test information is processed layer by layer into the test script required for the test, and the automated test is completed based on the automatically generated test script.

[0133] Specifically, the solution proposed in the present disclosure can be applied in a variety of user scenarios. Referring to Figure 1, a schematic diagram of user scenarios to which the present disclosure can be applied is shown. As shown in Figure 1, it can be applied to the testing of web pages. Specifically, each element in the web page can become the target object to be tested. Elements include controls and graphics, such as login controls, page jump controls, search controls, etc.; it can also be applied to the hardware performance testing of terminal devices. If the terminal device is a touch display, the touch function of the touch display can be the target object. If the touch display allows writing, its writing function can also be the target object to be tested.

[0134] During the test, the tester can upload test information for the target object to the first device and view the test results.

[0135] First, the present disclosure provides a first device, which is a device for executing the above-mentioned fully automatic testing process, as shown in Figures 2 and 3, Figure 2 shows an environmental architecture diagram of the first device in an automatic testing environment, and Figure 3 shows a schematic diagram of the step flow of automatic testing. As shown in Figures 2 and 2, the first device includes a processor, and a program runs in the processor. When the program is run by the processor, the target object on the first device or the second device is automatically tested.

[0136] Among them, the target object can be located in the first device or the second device, the second device can be a device independent of the first device, the first device can be a PC, a laptop, a server, a smart phone, a tablet, and the second device can be a PC, a server, a laptop, a smart phone, a tablet and a large all-in-one display.

[0137] As shown in Figure 2, the first device can be connected to a second device via a communication connection, such as a broadband network connection, a Bluetooth connection, or a signal line connection. The first device can also be connected to an external use case source via a communication network, which can be located on another server or another terminal device. The first device can receive test information, which can be input by a user via an input device or sent to the first device via a communication network by an external use case source.

[0138] As shown in FIG2 , a program is running in the processor, and the program can be stored in the memory of the first device. When it needs to be run, it can be run by the processor, and then the automatic test of the target object is performed on the first device; or, the program for performing the automatic test can form a readable storage medium and be located in the cloud, that is, located on a server connected to the first device, which can expose a calling interface to the first device. When the first device needs to perform an automatic test, it can receive test information and request to call the cloud interface, so that the cloud program can be called, so that the cloud program can perform parsing and mapping of the test information, and the program running in the processor of the first device can automatically test the target object based on the mapped operation command, thereby enabling the first terminal to automatically test the target object located in this device or other devices according to the cloud program. Specifically, the process of the automatic test performed can be shown in FIG3 , which specifically includes the following steps:

[0139] Step S101: Acquire test information sent to a first device, where the test information is used to describe the test content to be performed on the target object.

[0140] The test information obtained by the first device can be provided via a local text file, a database, or an external platform. Specifically, the test information can be uploaded by the tester or sent to the first device by an external use case source. The external use case source can be understood as a database located on another device, such as a server, a personal computer, or a laptop. The database can store multiple test information. In this way, when a function of the target object needs to be tested, the tester can call the use case source interface on the first device, and then retrieve the corresponding test information from the use case source. This test information can then be sent from the use case source to the first device.

[0141] The test information can be user-defined or filled in according to a standard template. Specifically, the test information can be textual, such as in txt or doc formats, or graphical, such as in excel. In some cases, the test information can even be image information, such as an image captured by taking a screenshot of the edited textual test information, where the image area containing the test information is included. In general, the test information can be edited by the tester based on their own preferences. This allows the tester to clearly describe the target object and test content during the test process, without having to worry about the test script that implements the test.

[0142] Among them, the test information can be used to describe the target object and the test to be performed on the target object, and specifically can include the description information of the target object, the test type to be tested on the target object, and the configuration information required for the test. Specifically, the description information can be text information or image information, which can include the name of the target object, the location in the second device, appearance information, URL (uniform resource locator) address, etc. For the URL address, the target object can be a web page. In this case, it can be a test for the web page, such as testing the frame rate, refresh rate and display integrity of the web page code; for appearance information, it can be an image of the target object or a description of the appearance characteristics of the target object. For example, if the target object is a writing page that can be written, the appearance characteristics description can be: in the whiteboard area on the display screen, if the writing page is provided with a grid under the writing function, the appearance characteristics description can also be added with the description of "there are grid lines in the whiteboard area". For another example, if the target object has a specific icon, the appearance characteristics description can also include the name of the icon. Assuming that the target object is a shopping app, the specific icon is a pig, then the appearance characteristics description can include the description "the icon is a pig".

[0143] The test type in the test information can represent the function of the target object to be tested. The target object can be a login object in a web page. The function to be tested can be a function for logging in after the user enters a username and password, or a function for verifying the username and password after the user enters the username and password during the login process. Specifically, in the test information, the test type can be replaced by a label, such as "login" can be replaced by the label "01". Of course, to facilitate editing by testers, the test type can be a Chinese name, an English name, or a name in another language, such as a Japanese name, without limitation.

[0144] The configuration information in the test information can indicate the parameters that the input device needs to input to the target object during the test, as well as the type of operation performed on the target object. The parameters input to the target object can include text, characters, etc., and these parameters can be included in the test information. The operation type can indicate the type of action performed by the input device on the target object during the test, such as sliding, single-clicking, and double-clicking. In the test information, the operation type can be represented in text or English.

[0145] The description information of the target object in the test information, the test type and configuration information of the target object can represent the target object to be tested, a function of the target object to be tested, the configuration information to be used to test the target object, and the actions that the input device needs to perform on the target object during the test.

[0146] For example, assuming that the test information is text information, and its content is "Open the browser, enter the address http: / / 10.251.×××, and press Enter", then in the test information, the test type can be the type of web page opened, the target object is the browser, and the configuration information is "http: / / 10.251.×××" and "Enter", where the operation type is "Enter" and the parameter is "http: / / 10.251.×××".

[0147] As another example, assuming that the test information is text information, and its content is "Enter user name [0-16], enter password [8-16, ×××]", "click to log in", then in the test information, the test type can be the login type, the target object is the login control, and the configuration information is "Enter user name [0-16], enter password [8-16, ×××]" and "click", where the operation type is "click" and the parameters are "user name [0-16], password [8-16, ×××]".

[0148] Step S102: parsing multiple key texts related to the test from the test information, and converting the multiple key texts into action sequences of the target data structure;

[0149] The action sequence is used to represent the operation process performed on the target object when executing the test content.

[0150] Among them, whether the test information is text-type information, chart-type information or image-type information, it can carry the above-mentioned description information, test type and configuration information, and the test information can be parsed to obtain the key text in the above-mentioned description information, test type and configuration information. Specifically, if it is text-type test information, the key text can be extracted from the test information. If it is table-type test information, the text information can be extracted from the table-type test information first, and then the key text can be extracted from the text information. If it is image-type test information, the text in the image can be recognized first, and then the key text can be extracted from the recognized text.

[0151] Specifically, the key text may refer to keywords in the description information, test type and configuration information. For the description information, the keywords describing the target object may be extracted from the description information. For example, if the verb and noun appear together in the description information, such as "open the browser", the key text of the description information is "browser"; for the test type, the key text may be extracted from the paragraph of the test information including verbs and nouns. For example, if "open the browser", the key text of the test type may be the type of "browser open"; for the configuration information, the key text may include the key text of the action type and the key text of the parameter type. For example, if the configuration information is "enter user name [0-16], enter password [8-16, ×××]" and "click", the key text of the action type may be "click", and the key text of the parameter type may be "user name [0-16], password [8-16, ×××]".

[0152] In one example, after parsing the test information, the multiple key texts obtained can be divided into key text describing the target object, key text describing the input, and key text describing the operation action. In other words, the multiple key texts parsed are all closely related to the test of the target object, and subsequent automated testing needs to be carried out according to these key texts.

[0153] After obtaining the aforementioned multiple key texts, the multiple key texts can be constructed into an action sequence that conforms to a target data structure. The target data structure can be pre-specified or adapted to different test types, such as different test types having different target data structures. The target data structure can indicate the order in which the multiple key texts are arranged, and the action sequence can be understood as a combination of the multiple key texts in a certain order.

[0154] In one example, the arrangement order can be the order of operations to be performed by the target object during the test process; the operation order refers to the order of actions to be performed by the user when holding the input device to test the target object. For example, in the test information of opening the browser, the user needs to first open the browser, then enter the network address in the browser, and then press Enter. The operation sequence is "double-click the browser → enter the address → press Enter".

[0155] In another example, the order of arrangement can be based on the subject-verb-object relationship between multiple key texts, thereby constructing a "verb + noun + parameter" data structure, and thus constructing an action sequence of "action + object + parameter". This subject-verb-object relationship can be obtained based on the parts of speech between multiple key texts, such as nouns following verbs and parameters following nouns. For example, in the login test information, the user clicks the login control and needs to enter the username and password in the login window during the process. The action sequence can be "click → login → username and password".

[0156] Of course, there may be other types of target data structures, which will not be described here.

[0157] Step S103: Mapping the action sequence into an operation command adapted to the target object, and executing the operation command on the target object to complete the test indicated by the test information on the target object.

[0158] In this embodiment, after obtaining an action sequence, the action sequence can be converted into an operation command adapted to the target object, where adaptation means that the operation command can be recognized by the device where the target object resides and is compatible with the system used by the device where the target object resides. During mapping, each key text in the action sequence can be mapped to a command line, where the programming language used in the command line is adapted to the system language of the device where the target object resides, thereby being callable and recognized by the device where the target object resides. Multiple key texts can have corresponding command lines, and these command lines are combined to form the operation command.

[0159] Among them, the interface where the target object is located can be called, and the operation command can be sent to the device where the target object is located through the interface, so that the device can execute the operation command on the target object. When executing the operation command, the device where the target object is located can simulate the input device to input parameters to the target object and then perform the corresponding operation action, thereby completing the test of the target object.

[0160] For example, taking the target object being located in the second device and being an object in a web page as an example, assuming that the second device adopts the Android system, the test information is "enter user name [0-16], enter password [8-16, ×××]", "click to log in", and the action sequence corresponding to the test information is "click to log in user name and password". The action sequence can be mapped to the operation command under the Selenium / Pyautogui framework that supports the Web end in the Android system. Then, the second device inputs the user name [0-16] and password [8-16, ×××] in the login window by simulating the mouse, and then simulates the mouse clicking the login control, thereby completing the login test of the login window.

[0161] As another example, taking the target object being located in the second device and the target object being a browser as an example, assuming that the second device uses the Android system, the test information is "open the browser, enter the address http: / / 10.251.×××, and press Enter", and the action sequence corresponding to the test information is "Enter the browser http: / / 10.251.×××". The action sequence can be mapped to an operation command under the Selenium / Pyautogui framework that supports Web use in the Android system. Then the second device simulates the mouse to enter http: / / 10.251.××× in the login window, and then simulates the keyboard Enter, thereby completing the test of opening the page in the browser.

[0162] Among them, the operation command can be regarded as an input operation performed by a virtual input device on the target object, and the first device can be understood as an external input device, which performs positioning, input and triggering operations on the target object through the operation command, a virtual input device. Therefore, without the help of the input device of the device where the target object is located, the input device can be simulated remotely or locally using computer language to manipulate the target object and complete automatic testing.

[0163] Using the first device of this embodiment, the first device will parse the received test information, convert the parsed key text into operation commands for operating the target object, and call the interface of the target object to execute the operation commands on the target object, thereby automatically completing the test operation on the target object; in this way, through parsing and mapping, the user-defined test information can be directly converted into executable operation commands, thereby, the tester does not need to write a test script, thereby reducing the requirements for the tester; compared with writing a script, the time length of customizing the test information is greatly shortened, thereby improving the efficiency of automatic testing, and then the manpower cost and time cost required for inspection can be greatly reduced.

[0164] Among them, the environmental architecture for realizing automatic testing of the first device is explained. Specifically, as shown in Figure 1, it can include an algorithm library, a test case library and a variety of library files, wherein the algorithm library and the various library files can be located in the memory of the first device, and the various library files can include an object library (hereinafter referred to as the third library file), an action library (hereinafter referred to as the second library file) and a test database (hereinafter referred to as the first library file); specifically, the algorithm library can parse the test information, analyze the test results, etc., and the various library files can serve as support for mapping action sequences into operation commands. Specifically, it can also include an interface layer between the second device where the target object is located.

[0165] Referring to FIG4 , a diagram of the operational architecture of the first device for automatic testing is shown. As shown in FIG4 and FIG1 , it includes a use case layer, a parsing layer, a mapping layer, an execution layer, and a supervision layer. The use case layer includes multiple use case sources, each of which can store multiple test information uploaded by users. In other words, testers can search for the required test information from the use case source. Different use case sources can be located on different devices, and the format types of the test information stored in different use case sources can be different, such as one use case source storing text-type test information and another use case source storing table-type test information. In another exemplary embodiment, the test information stored in different use case sources can correspond to different target objects, such as the test information in one use case source is all for elements in a web page, while the test information in another use case source is all for electronic components in a terminal device. Alternatively, the test information stored in different use case sources can correspond to different test functions. For example, the test information in one use case source is used to test functions related to login, such as login, username verification, password verification, etc., the test information in another use case source is used to test functions related to web browsing, and the test information in yet another use case source is used to test the writing function on a touch screen.

[0166] The basic fields for use case writing must be included to ensure the readability and reviewability of the test case and meet the input requirements of intelligent testing. The following example shows

[0167] Among them, the algorithm library is located in the parsing layer, which is used to extract multiple key texts related to the test from the test information and combine the multiple key texts into an action sequence of the target data structure; in the mapping layer, it includes a first library file, a second library file and a third library file, wherein the three library files are used to map different types of texts in the multiple key texts, as shown in Figure 4, including action mapping, object positioning and rule matching, wherein action mapping refers to mapping the key text of the action type with the operation hole file, object positioning refers to mapping the key text of the object attribute with the object library file, and rule matching refers to mapping the key text of the element attribute (input parameter) with the test database file, thereby realizing the mapping of the three main key texts related to the test into operation commands;

[0168] At the execution layer, the operation command is passed to the target object, thereby simulating the operation of the input device on the target object. The process of the target object responding to the operation command is a test process, and the result of its response to the operation command is the test result.

[0169] As shown in FIG4 , the architecture may further include a supervisory layer, which may monitor the process of executing the operation command on the target object when calling the interface of the target object to execute the operation command. For example, the supervisory layer may obtain the operation log of executing the operation command on the target object, the page source code, and the display changes of the page where the target object is located during the execution process, etc., so as to determine whether the test result meets expectations and automatically give the test result. For an introduction to the supervisory layer, please refer to the subsequent exemplary embodiments and will not be elaborated on here.

[0170] In combination with the above architecture, some exemplary embodiments of the automatic test disclosed herein are further described.

[0171] In some examples, when parsing test information using an algorithm library, key text in the test information can be first extracted, and then the key text can be structured to obtain an action sequence. Specifically, when parsing the test information, the test information can be first segmented, and then unnecessary words can be removed from each segment to obtain multiple key texts. The multiple key texts can then be standardized to obtain an action sequence.

[0172] In specific implementation, the test information may be segmented to obtain multiple paragraphs; and based on a preset blacklist and / or whitelist, key text may be extracted from each paragraph.

[0173] In this example, if the test information is text-based information, the test information can be directly segmented according to punctuation marks to obtain multiple paragraphs; if the test information is table-based information, the text information in the table needs to be extracted and then segmented according to the original positions of the text information in the table to obtain multiple paragraphs; if the test information is image-based information, the text areas in the image can be recognized to obtain multiple text information, and then the text information can be segmented according to the text areas to obtain multiple paragraphs. For example, in an image, a paragraph of text occupies different areas in the image, and there is an interval between the area where one paragraph of text is located and the area where another paragraph of text is located. Therefore, the recognized text information can be segmented according to the image areas recognized by text recognition to obtain multiple paragraphs.

[0174] Among them, for each paragraph, useless words can be removed from the paragraph and useful words can be retained to obtain the key text. Specifically, based on a preset blacklist, the useless words in the paragraph can be removed. The preset blacklist can include multiple useless words. For example, it can include auxiliary words such as "de", "de", "zai", "jie zhe", etc. The words in the paragraph can be matched with the multiple words in the blacklist, and the matched words can be deleted from the paragraph. Or, based on a preset whitelist, multiple key texts can be directly screened out from the paragraph. The whitelist includes multiple useful words, and the multiple useful words are words that are closely related to the test, such as words with a high usage frequency, such as words with a high usage frequency like "login", "click", "enter", etc. Thus, the words in the paragraph can be matched with the multiple words in the whitelist, and the matched words can be screened out from the paragraph as key texts.

[0175] Another example is to adopt a combination of the whitelist and the blacklist. For example, first, based on the blacklist, the useless words are deleted from the paragraph. Then, based on the whitelist, the useful words are retained from the paragraph after deleting the useless words. After that, for the remaining text in the paragraph, the short sentences and phrases in the text can be screened out as key texts, thereby avoiding missing the screening of key texts.

[0176] In another example, the key text can be parsed based on a pre-trained neural network, such as inputting test information into the neural network, and outputting each key text through the neural network; wherein, the neural network can be a network using a self-attention mechanism, such as a Transformer network, which can encode and embed the test information, and then perform feature extraction on the encoding after word embedding, extract word vectors, and then construct attention scores between each word vector. According to the attention scores, the correlation between phrases in the test information can be determined, that is, the semantic correlation between each phrase in the test information can be obtained, and thus, based on this correlation, multiple key texts can be output. In a further example of this example, the neural network can also output the attributes of each key text, such as whether it belongs to an element attribute, an object attribute, or an action attribute.

[0177] Among them, the process of training this neural network can be: collecting multiple test information samples, each test information sample corresponds to multiple standard key texts, and the annotated key texts participate in the supervision of training as labels of the test samples. Multiple standard key texts can be completed through manual annotation or extracted through blacklists and whitelists; encoding multiple test information samples and inputting them into the neural network, constructing a loss function based on the predicted key text output by the neural network and the standard key text, and updating the parameters involved in the network. After multiple updates, the neural network training is completed and can be applied to the reasoning stage.

[0178] Among them, when using blacklists and whitelists to parse test information, the blacklists and whitelists can be stored in an algorithm library; when using neural networks to parse test information, the neural network can run in the processor of the first device, or be stored in the memory of the first device, and can be called by the processor when needed.

[0179] Among them, when using blacklists and whitelists to parse test information, the accuracy and comprehensiveness of key text screening can be improved by continuously updating the words in the blacklists and whitelists; when using neural networks to parse test information, the generalization performance of test information parsing can be improved, and the standardization requirements for test information can be reduced, thereby increasing the user's freedom to edit test information and optimizing the user experience.

[0180] For example, the test information is assumed to include: opening a browser, entering the address [http: / / 10.251.186.5 / # / login], and pressing Enter; entering a user name [D~16], a password [8~16,xx], and clicking Login; and assertion information (a description of the expected test result): displaying text.

[0181] Then we can first perform word segmentation and get [pair[open, V], pair(browser, 'm)],

[0182] [pair(input, V],

[0183] pair(address,'n),

[0184] pair (T,

[0185] [pair(Enter, V]

[0186] [pair(input, V)

[0187] pair(username,'n),pair(T,X),pairO,x],pair(~,x),pair(16,'m?,pair(T,x)]

[0188] [pair(input, V].

[0189] pair(password,'n)

[0190] pair(T,'x),pair(8,x),pair(~,X),pair(16,'m),pair(;,'X),pair(xxx,'eng),pair(J,x)

[0191] [pair(input; V), pair(verification code, 'n), pair(T, x), pair('t, x), pair(T; x]

[0192] [pair[login, V]];

[0193] After parsing each word, the part of speech of each word can be determined. Then, the parsed words can be combined to obtain an action sequence that conforms to the target data structure:

[0194] For example, action sequence A: [faction: open; target: browser, params: "?, [action: input, target: "address, params?" "http / / xxXXXXXX / login], {action?: "Enter, target:", params:"]}

[0195] For example, action sequence B: {action: input, target: "username", params? "0~16], [action': input; target: password, params? "8~16xxx], [action': input; target: verification code; params: "1], [action' login, target: ", params: "].

[0196] In some exemplary embodiments, the first device may include a memory, wherein the above-mentioned multiple library files may be located in the memory and may be called during the parsing of the test information. Specifically, the multiple key texts parsed from the test information may be mapped to the corresponding library files respectively, thereby outputting the corresponding command lines. These command lines are combined in their operation order to obtain an operation command, which can also be understood as obtaining a test script. Each library file has pre-stored preset command lines corresponding to different keywords under the same attribute. The attribute can be used to indicate the part of speech of the key text, such as whether it is a verb or a noun. Generally speaking, the key text of the verb type indicates a certain type of operation on the target object, such as a click operation, a double-click operation, etc.; the key text of the noun type is generally used to indicate the operation parameters and target object required in the operation. For example, in an input operation, the key text can be input data that needs to be typed in, such as a login username and password; for example, in the description information of the target object, it can be used to describe the name, appearance and location information of the target object. For example, if the target object is a "search" element in a web page, the description information may include "search element", which is generally used in conjunction with the verb of the operation type.

[0197] In specific implementation, different types of library files can correspond to key texts of different attributes in the action sequence. When mapping the action sequence into operation commands adapted to the target object, each key text in the action sequence can be mapped to the library file corresponding to the attribute of the key text to obtain the command line corresponding to the key text; based on the command lines corresponding to multiple key texts, an operation command is generated; wherein the library file includes preset command lines corresponding to multiple keywords under the same attribute.

[0198] In this example, the decomposed multiple key texts can obtain their own attributes, which can be used to characterize the operation object corresponding to the key text during the test. For example, for a test, it can include three attributes, object attributes, element attributes and action attributes, where the key text of the element attribute can refer to the text related to the input in the configuration information, such as the user name and password, that is, the key text of the element attribute is the input parameter relied on during the test; where the action attribute can be the text related to the action in the configuration information, which can characterize the triggering operation performed on the element or interface, and the triggering operation can be enter, single click, double click, slide and other actions; the object attribute can be the key text in the description information of the test information, such as "login", "browser", and the key text of the object attribute can accurately identify the target object.

[0199] After multiple key texts form an action sequence with a certain arrangement order, each key text in the action sequence can be mapped to a corresponding library file according to its attributes. Since the library file includes preset command lines corresponding to multiple keywords, the key text can be compared with the keywords in the library file to locate the preset command line where the key text is located, and then use the preset command line as the command line for the keyword. The multiple preset command lines included in the library file of each attribute can all be command lines of the same attribute, and the preset command lines corresponding to different keywords can be different.

[0200] For example, the various preset command lines included in the library file of the action attribute are all command lines for the input device to perform a certain operation action. For example, the trigger operation performed on an element or interface can be attributed to the library file. The trigger operation can be enter, left mouse button click, right mouse button click, left button double-click, right button double-click, and sliding with the left button held down. Specifically, different operation actions may require the operation of different input devices, so they can correspond to different preset command lines, such as enter corresponding to a preset command line, and left mouse button click corresponding to a preset command line. Therefore, when mapping the key text of the action attribute to the library file of the action attribute, the key text can be compared with the keyword corresponding to each preset command line to obtain the command line corresponding to the key text.

[0201] For example, the various preset command lines included in the element attribute library file are all command lines for the input device to execute to input certain parameters. Since different target objects may input parameters in different ways, the purpose attributes of the input parameters may also be different. Therefore, the multiple keywords included in the meta attribute library file can correspond to different purpose attributes. For example, the keyword "username" corresponds to the purpose attribute of inputting the username, and the keyword "password" corresponds to the purpose attribute of inputting the password. When mapping the key text of the element attribute to the element attribute library file, the key text can be compared with the keyword corresponding to each preset command line to obtain the command line corresponding to the key text.

[0202] Specifically, when mapping the key text of the element attribute to the library file of the element attribute, when obtaining the corresponding preset command line, the content of the key text of the element attribute can be passed into the matched preset command line to obtain the command line of the key text.

[0203] For example, the various preset command lines included in the library file of object properties may include key values ​​of different objects, wherein, when the target object needs to be tested, it is first necessary to locate the target object on the device, and by mapping the key text of the object property to the library file of the object property, the positioning command of the target object on the device can be obtained, that is, the library file of the object property can help locate the target object and return the command line for locating the target object; in one example, the preset command line in the library file may be the key value of the target object, that is, what is returned is the key value of the target object on the device, and the target object can be accurately located through the key value. Generally speaking, different target objects on the device may correspond to different key values,

[0204] That is to say, the command line obtained by the above mapping can be a program code or a key value, and the multiple command lines obtained by mapping multiple key texts can include program code or key values. Therefore, after obtaining multiple command lines, the multiple command lines can be combined into operation commands according to the arrangement order of the action sequence, or operation commands can be generated based on multiple command lines according to the underlying framework adapted to the target object.

[0205] In this example, for each attribute, there can be multiple library files, and different library files can correspond to different operating systems. For example, for the object library file of the object attribute, it can include an object library file adapted for the Android system, and it can also include an object library file adapted for the DOS system, so that for different systems, it can obtain a mapping adapted for the system.

[0206] In this example, the various library files can be regarded as the representation of the various functions of the input device in computer language, that is, the numerous command lines in the library files simulate the numerous functional operations of the input device. For example, for the library files of action attributes, the numerous operations that can be performed by the mouse and keyboard can be simulated, and the physical operations are converted into operations expressed in virtual computer language; for the library files of object attributes, the positioning operation of the mouse can be simulated, and the positioning of the mouse cursor can be converted into positioning expressed in virtual computer language; for the library files of element attributes, the typing operation of the keyboard can be simulated, and the keyboard typing is expressed in virtual computer language.

[0207] Therefore, when mapping the action sequence based on the library file, it is equivalent to converting a series of operations performed by humans holding input devices into corresponding computer language representations. The generated operation commands represent the operations performed by the input devices, which can also be understood as converting the operations of the physical input devices into the operations of the virtual input devices. Therefore, when the operation commands reach the target object, it is equivalent to the physical input device performing a series of operations on the target object, so that the target object can be directly controlled remotely or locally without the help of the input device of the device where the target object is located.

[0208] For example, the library file can be matched with the system that supports the target object. For example, if the target object is a web page, the library file used can be the Selenium / Pyautogui framework. For another example, if the target object is a writing page on a terminal device, the library file used can be determined by the terminal system. If the terminal system is Android, the library file can use the Uiautomator2 framework.

[0209] In some examples, the attributes include action attributes, element attributes, and object attributes, and the key text of the object attributes is used to identify the target object; wherein the multiple library files include an action library file corresponding to the action attributes, a test database file corresponding to the element attributes, and an object library file corresponding to the object attributes;

[0210] The action library file includes preset command lines corresponding to various operation actions, the test database file includes preset command lines corresponding to different input parameters, and the object library file includes preset command lines for locating different objects.

[0211] In this example, the action attributes, element attributes and object attributes of the key text can be determined according to the contribution made by each key text in the action sequence during the test process. For example, the contribution made by the key text of the action attribute is to trigger the state change of the target object, such as page jump, which causes the page display state to change; and the contribution made by the key text of the element attribute is the prerequisite for entering the target object in this test, such as in the login test, typing the user name and password is a prerequisite; and the contribution made by the key text of the object attribute is to locate the target object in the first device or the second device; therefore, according to the contribution made by each key text in the test, the attributes are divided into action attributes, element attributes and object attributes.

[0212] Of course, in some other examples, the operational process involved in a test may be very complex, which involves more attribute divisions. For example, in a writing test, it also includes the planning of the writing trajectory. That is to say, the planning of the trajectory points is also a prerequisite for the test. The element attributes can be divided into pre-element attributes and post-element attributes. The trajectory point planning can belong to the pre-element attribute, and writing according to the trajectory points can belong to the post-element attribute.

[0213] This example uses three attributes as examples. Each attribute corresponds to a library file, such as the action attribute corresponding to the action library file, the element attribute corresponding to the test database file, and the object attribute corresponding to the object library file. As described in the above examples, the action library file can include preset command lines corresponding to multiple action keywords, the test database file can include preset command lines corresponding to multiple input purposes, and the object library file can include key values ​​corresponding to multiple objects.

[0214] 5 , a schematic diagram of the process of mapping an action sequence is shown. In one example, multiple key texts may include a first text of an element attribute, a second text of an action attribute, and a third text of an object attribute. Accordingly, as shown in FIG5 , in the step of mapping an action sequence into an operation command adapted to a target object, the first text may be mapped to a test database file to obtain a first command line containing keywords of the first text; the second text may be mapped to an action library file to obtain a second command line containing keywords of the second text; the third text may be matched with an object library file to obtain a key value corresponding to the target object; thereby, the first command line, the second command line, and the key value are combined in a certain order to obtain an operation command; the operation command may be recognized and operated by the device where the target object is located.

[0215] In some examples, the object library file may also include module names, object action descriptions, object image information, and object text information of various objects, so that the target object can be located based on the key text of the object attributes, as well as the module names, object action descriptions, object image information, and object text information of various objects. That is, the key text of the object attributes is matched with the module names, object action descriptions, object image information, and object text information of various objects to determine the target object.

[0216] This example divides the key text in the action sequence into three attributes. This allows multiple key texts to be divided according to their contributions to the test process, thereby building a clear computer language framework for simulating input devices. This makes the mapping more consistent with the actual usage habits and operation sequence of the input device.

[0217] In some examples, as shown in Figure 1, the first device can be applied to a variety of user scenarios, including, for example, testing of terminal applications and testing of terminal equipment. In the testing scenario of terminal applications, the test object can include a web page, such as testing elements in a web page, that is, the target object located in the web page can be tested; for another example, the target object can be a writing page, and the writing page can be used to provide writing. Specifically, the device where the target object is located can include a touch screen to support writing on the writing page, thereby testing the writing function of the writing page.

[0218] In which, when the target object includes a writing page, the writing function of the writing page can be tested. In one example, when calling the interface where the target object is located and executing an operation command on the target object, the interface of the writing page can be called to execute virtual writing on the writing page according to the operation command line, or the interface of the mechanical device can be called to execute physical writing on the writing page according to the operation command line.

[0219] In this embodiment, the writing page may refer to a display page included in the writing application for users to write. Generally speaking, the device where the writing page is located needs to have a touch screen. For example, if the writing page is in the first device, the first device has a touch screen, and if the writing page is in the second device, the second device also needs to have a touch screen to support the writing function of the writing page.

[0220] Among them, physical writing may refer to the first device sending an operation command to the mechanical device, so that the mechanical device touches the writing page, thereby realizing the writing function on the writing page; in this case, the writing page may be in a writing function state; wherein, the mechanical device may be a robotic arm, specifically, a six-axis robotic arm, that is, a robotic arm that can move freely in six directions; then the operation command may include trajectory point data indicating the movement trajectory of the robotic arm, that is, including the posture data of the robotic arm at each trajectory point, so that the robotic arm can draw a corresponding graphic on the writing page according to the trajectory indicated by the trajectory point data in the operation command. The graphic is the graphic specified during the test, and the accuracy of the writing page's response to the user's writing is tested through the line width, trajectory, color, etc. of the graphic.

[0221] In this example, the mechanical device can establish a socket communication connection with the first device to ensure communication stability between the mechanical device and the first device.

[0222] In this example, virtual writing may refer to the device where the target object is located simulating the input device writing on the writing page. Since the touch screen is not actually touched, this virtual writing may refer to transmitting the track point data to the interface where the writing page is located, packaging the track point data into touch point data, thereby "tricking" the writing page into generating a writing track according to the track point data.

[0223] In which, when the target object includes an object in a web page, when calling the interface where the target object is located and executing an operation command on the target object, the first device can call the interface where the web page is located to simulate the input device of the device where the target object is located to perform an operation corresponding to the operation command on the target object.

[0224] In this example, the target object to be tested can be an object on a web page, including various elements and input fields. In some cases, if the refresh test includes a web page, the target object can be a refresh element, such as the refresh key on the keyboard or the refresh element in the pop-up box when clicking the right mouse button. In this case, the refresh element does not belong to the web page. In this case, the target object can refer to the object that controls the display of the web page. In other words, in some examples, the target object can include an object on the web page or an element that controls the display of the web page, and both can be used to perform corresponding operations on the web page.

[0225] Among them, since the target object is an object used to perform operations on a web page, when testing the target object, the interface where the web page is located can be called, and the simulated input device can perform an operation corresponding to the operation command on the target object, so that the target object completes a certain operation on the web page. For example, if the target object is a login window, after the simulated input device enters the user name and password in the login window and clicks login, the web page can jump to the interface after login; if the target object is a refresh element, after the simulated input device performs a refresh operation in the web page, the web page can be refreshed, thereby displaying the refreshed interface.

[0226] Among them, the interface where the web page is located can refer to: the API (Application Programming Interface) exposed to the outside by the web page in the device where it is located, and its calling process can be: pre-establishing a calling relationship between the first device and the API, the first device sends the operation command to the device where the target object is located (the first device or the second device), and then the device calls the interface and passes the operation command into the interface, that is, simulating the input device to pass the operation parameters to the target object, so that the target object responds to the operation command as if it responds to the operation of an external input device, thereby completing the test of the target object.

[0227] Accordingly, in the test scenario of the terminal application, the target object can be an object in a web page, such as various elements in a web page. Then, before executing the corresponding operation command on the target object, the target object can also be located in the web page. Since a web page generally includes a large number of elements, when designing the test information, it will take a lot of time to determine the location of each element separately. Therefore, in this example, an automatic positioning method can be used to determine which object in the web page the test information is. In this way, the target object to be tested can be automatically located in the web page based on the descriptive information in the test information. Specifically, when the program is run in the processor, it is also used to locate the target object, wherein the process of locating the target object can include: obtaining identification information of each object in the currently displayed web page; and matching the target identification information with multiple identification information to obtain the position of the target object in the currently displayed web page.

[0228] In this embodiment, before executing an operation command, the target object needs to be located. That is, in the process of mapping an action sequence to an operation command, the location of the target object in the web page needs to be known so as to obtain its key value in the web page, thereby facilitating subsequent testing operations. Since the test information includes descriptive information describing the target object, which may include the name, location, or appearance of the target object, the description of the target object can be referred to as the target identification information of the target object, which can identify the target object.

[0229] In order to accurately locate the target object in the web page, the identification information of each object in the current web page, such as the identification information of each element, can be obtained. In specific implementation, the interface of the device where the web page is located can be called to obtain the UI data source code of the web page, and the identification information of each element can be obtained based on the information of each element in the UI data source code; of course, the identification information of each element can also be obtained in other ways, such as obtaining the image of each element in the web page to obtain the identification information of the image type. Then, the target identification information is matched with the identification information of multiple elements to obtain the position of the target object in the currently displayed web page. Among them, the target identification information and the identification information belong to the same type, such as both belong to text type, both belong to image type, or both belong to encoding type.

[0230] Therefore, the target identification information can be the appearance image of the target object, and thus, the identification information can also be the appearance image of each element. When the two are matched, the similarity between the two can be determined, and thus the location of the target object can be determined based on the similarity.

[0231] Among them, the target identification information and the identification information can both be text information. For example, the target identification information is the name of the target object, and the element identification information can also be the name of the element. In this way, when comparing, the location of the target object can be determined based on the consistency between the names.

[0232] Exemplarily, since an object library file can be included and used to map key text of object attributes, in one example, identification information of each object in a web page can be stored in the object library file, thereby facilitating the location of the target object. Specifically, for an object, the object library file can store the object's image identification information, text identification information, and encoding identification information. The more types of identification information stored, the more diverse the types of descriptive information in the test information can be, allowing testers to customize the descriptive information as needed, thereby optimizing the tester experience and reducing the tester's workload.

[0233] In a further example, the positioning of the target object may include image-based positioning and name-based positioning. In image-based positioning, a screenshot of the currently displayed web page can be taken, and the screenshot image can be matched with the appearance image of the target object to obtain the position of the target object in the currently displayed web page. Alternatively, the text in the screenshot image can be recognized, and the recognized text can be matched with the name of the target object to obtain the position of the target object in the currently displayed web page. Either or both of these methods can be performed.

[0234] In a specific implementation, a screenshot of the currently displayed web page may be taken, and the screenshot image may be divided to obtain multiple image regions; wherein different objects correspond to different image regions; then, at least one of the following processes may be performed on the image regions to obtain identification information corresponding to each object:

[0235] The first processing method is to perform text recognition on the image area to obtain the identification information of the text type of the object to match it with the name;

[0236] The second processing method is to extract features from the image area and obtain identification information of the feature representation type to match it with the feature map of the appearance image.

[0237] In this example, an interface of a device where a web page is located can be called to take a screenshot of the web page, such as by transmitting an operation parameter to a screenshot button on the web page so that the device can take a screenshot of the web page, thereby obtaining a screenshot of the web page. The screenshot can include images of all elements on the web page; wherein, each image area in the screenshot image can be divided to obtain image areas where multiple elements are respectively located. When dividing each image area, each element in the screenshot can be first identified to obtain a position frame of each element. Then, the area selected by each position frame can be used as the image area of ​​the element. Specifically, the position frame of each element can be obtained using a neural network model. For example, the screenshot image can be input into the neural network model, and the neural network model can perform feature extraction on the screenshot image. Based on the feature map obtained by the feature extraction, the position frame of each element is determined.

[0238] The image area where the location box is located can be extracted from the screenshot image, thereby achieving image segmentation of the web page. By identifying the segmented image area, identification information of each element can be obtained.

[0239] Specifically, in the first processing, text recognition can be performed on the image area. Specifically, an OCR recognition method can be used to obtain text information in each image area, thereby obtaining element identification information. This text information is generally used to identify the name of the element. In this case, the target identification information of the target object can be the name of the target object. In this way, by comparing the name of the target object with the names of each element, the element that is consistent with the name of the target object is determined, and then the element is determined as the target object. When using this method, the descriptive information in the test information can be the name of the target object, so that the user only needs to enter the name of the target object to be tested.

[0240] In the second processing, feature extraction can be performed on each image area to obtain a feature map of each image area. The feature map can be used as identification information of the element. That is, the identification information of the element can be information of a feature representation type. Therefore, the description information in the test information can be the appearance image of the target image. During matching, feature extraction can be performed on the appearance image to obtain an appearance feature map of the target object. Therefore, the target identification information of the target object is also of a feature representation type. Then, the appearance feature map of the appearance image can be compared with the feature maps of each image area. Specifically, the similarity between the two can be calculated, so that the element corresponding to the feature map with the highest similarity is used as the target object. When using this method, the description information in the test information can be allowed to be the appearance image of the target object. Therefore, the appearance image can be carried in the test information or sent to the first device together with the test information. Then, the first device locates the target object to be tested in the web page through the appearance image and each image area in the web page.

[0241] The first and second processing methods can be combined. For example, if the position of the target object in the web page is obtained through the first processing, the position of the target object in the web page can also be obtained through the second processing. The two positions can be compared. If they are consistent, the position of the target object can be determined. If they are inconsistent, the position with the higher score can be used as the position of the target object. In this case, in the first processing, the name of the element in the image area and the name of the target object can have a matching score, while in the second processing, each feature map and the appearance feature map have a similarity score. Generally, in the first processing, the element with the highest matching score is used as the target object, and in the second processing, the element with the highest similarity score is used as the target object. When the two are consistent, it means that the accuracy of the target object determined is high. When the two are inconsistent, the higher of the matching score and the similarity score can be used as the target object.

[0242] In the first and second processing, each image area in the web page and the feature map of each element in the web page can be stored in the object library file. Specifically, when the first device completes the initialization of the program, it can call the interface of the device where the web page is located to obtain a screenshot of the web page, thereby completing image segmentation and image feature extraction, thereby obtaining the image area and feature map of each element in the web page, and saving them in the object library file. Therefore, when performing the first and second processing, it is sufficient to directly search for the various image areas and feature maps of the currently displayed web page from the object library file.

[0243] Specifically, since it is necessary to obtain the feature map of the elements in each web page, in this example, when the processor initializes the program, the UI design drawing of the web page can be obtained, and then the image of each element in the UI design drawing can be segmented, and each image area obtained by the image segmentation can be saved, and features can be extracted from each image area to obtain the feature map of each element. The feature map of each element and the image area without feature extraction can be saved in the object library file. During the test process, after obtaining the image of the current web page, the UI design drawing corresponding to the current web page can be searched in the object library file, and then the feature map and image area corresponding to each element in the found UI design drawing can be extracted, and then the first and second processing described above can be performed.

[0244] Of course, in some other embodiments, in order to improve the efficiency of locating the target object, a combination of multiple processing methods can be used. The method that first outputs a matching result is used to determine the target object. For example, the multiple methods can include the first processing method, the second processing method, and a directory query method, wherein the directory query method is to search the page directory of the web page for the target object corresponding to the description information.

[0245] In some examples, if it is necessary to perform parallel testing on target objects in web pages of multiple terminal devices, the location of the target objects can also be performed through a file query method, which can include the first processing method and / or the second processing method. In this way, by locating the target object in the web page of one terminal device, the location of the target object in the web pages of other terminal devices can be obtained.

[0246] Of course, in some other embodiments, as described above, the descriptive information in the test information can also include the name, ID, xpath, css, etc. of the target object to complete the search and positioning. In this case, the first device can call the interface of the web page to obtain the page data of the web page, and then complete the search and positioning of the target object through the xpath, css, id, and name of each element in the page data.

[0247] Among them, after determining the target object to be tested on the web page, the name of the target object on the web page can be returned to the object library file. The object library file also saves the correspondence between the position and name of each object in the web page, so that your object library file can return the position of its target object in the web page. The position can be used as the key value of the target object to form subsequent operation commands.

[0248] Specifically, when the key value is the position of the target object in a web page, the key value may include the page coordinates of the target object. When testing the target object, the page coordinates can be provided to the device where the target object is located, and the device simulates the movement of the cursor so that the cursor moves to the position of the target object, thereby performing a test on the target object at that position.

[0249] As shown in Figure 1, a test scenario for a writing page can also be included. In this test scenario, the target object can be a writing page, and the device where the writing page is located includes a touch screen display. The touch screen display provides hardware support for writing on the writing page. During the test, it is possible to test whether the writing function of the writing page is normal and whether it can accurately respond to writing under specific settings. Referring to Figure 6, a schematic diagram of a scenario for automatically testing the writing function of a writing page is shown. As shown in Figure 6, the target object can include a writing page, and the test information can include first test information for performing a writing test. Accordingly, in this test scenario, since it is necessary to simulate the input device writing on the writing page of the touch screen display, the track point data that the writing depends on can be prepared in advance. In this way, during the test, the input device can be simulated to write according to the track points provided by the track point data, and the written graphic can be regarded as an image formed by connecting the track points in the track point data. Accordingly, when multiple key texts related to the test are parsed from the test information, the display module of the first device can be controlled to display a preset writing setting page in response to the arrival of the first test information; the writing setting page is pre-set with multiple writing templates, and each of the writing templates includes at least one of the following setting items: line, line width, color, font, background; then, in response to the setting information received by the writing setting page, writing data and track point data are generated; thereafter, the writing data and track point data are determined as key texts.

[0250] Accordingly, when calling the interface where the target object is located and executing an operation command on the target object, the interface where the display component is located can be called to simulate the writing device to perform virtual writing on the display component, and / or call the mechanical device connected to the first device to perform physical writing on the writing page.

[0251] In the test of the writing page, the writing trajectory needs to be planned in advance, wherein the writing trajectory can be planned in the test information, but this increases the workload of the tester. In this example, a test label can be set for each test information to characterize its test category, and a writing test label can be set in the test information of the writing test. The label can be carried in a specific field of the test information. When the first device obtains the test information, it can first read the specific field and determine the type of test according to the characteristic field. If the test type is a writing test type, the preset writing setting page can be displayed on the display module of the first device in response to the type of the writing test.

[0252] Specifically, the writing setting page can be bound to the program, and the page data of the writing setting page can be stored in the memory. When the writing page setting page is displayed, the page data can be taken out from the memory and handed over to the display module, so that the display module can render the page data on the display screen of the first device. Among them, the writing setting page can allow users to set various configurations required for writing. Specifically, it can include various writing templates. Different writing templates correspond to writing different contents, such as writing templates including text, numbers, letters, formulas, graphic shapes, lines, etc. Among them, text refers to writing text on the writing page, numbers refer to writing numbers, letters refer to writing English letters on the writing page, and of course, letters in other languages ​​can also be included. Formulas are formulas that can be handwritten on the writing page. Graphics refer to graphics of various shapes that can be drawn on the writing page, such as rectangles, triangles, etc.; lines can refer to drawing various lines on the writing page.

[0253] Among them, for each writing template, multiple setting items can be provided for users to set which writing settings to use under the writing template, such as lines, line width, color, font and background, etc.; among them, the line can represent which stroke is used for writing. In this case, multiple pen shapes can be pre-set on the writing setting page for users to choose, such as markers, signature pens, brushes, pencils, etc. Of course, the line can represent which type of line is used, such as wavy lines, straight lines, dotted lines, etc., and the line setting item can be applied to any of the above writing templates; among them, the line width can represent the width of the line when a certain line is used, and can also be understood as the thickness of the line. Of course, the writing setting page can also preset multiple lines of different widths for users to choose from, or preset multiple pound values, each pound value corresponds to a line of a width, and multiple pound values ​​can be provided for users to choose from, and the line width setting item can be applied to any of the above writing templates. ; Among them, the color can represent the color of the written lines, and a variety of colors can be preset on the writing setting page for users to choose from. Of course, the more colors there are, the higher the accuracy of the writing test can be. The color setting item can be applicable to any of the above-mentioned writing templates; among them, when the font represents the written text, the font style of the text, such as Kaiti, Songti, and Roman fonts, can include fonts corresponding to Chinese and English, and can even include fonts corresponding to other languages; the font setting item can be applied to text templates, letter templates, and number templates; among them, the background can be the background of the writing page, which can include the color and shape of the background, such as the color can be selected from white background, green background, black background, and gray background, etc. Of course, a variety of background colors can be preset on the writing setting page for users to choose from; for example, the shape can be selected from grid, polka dot or other shapes, and even an image can be provided as the background, which will not be elaborated here.

[0254] Among them, each setting item in the writing setting page can allow the user to perform input operations. Specifically, referring to Figure 7, a page schematic diagram of a page template in a writing setting page is shown. As shown in Figure 7, after selecting the writing template, multiple options can be provided for the user to choose in each setting item of the writing template. The user can complete the configuration of the writing by selecting the corresponding option. For example, if the user selects the options of red lines, line width of 1 pound, font of regular script, and background of green, then the track point data for corresponding writing on the writing page can be generated based on the selected writing template and the setting options selected in the writing template.

[0255] Specifically, after the user sets various setting information on the writing setting page, a handwriting learning generation algorithm can be used to generate trajectory point data according to the selected writing template. Then, according to the selected setting items, writing data corresponding to the setting items selected by the user and having a certain style can be generated, and both can be used as key texts, such as as key texts of element attributes.

[0256] Exemplarily, the selected writing template is a graphic writing template, in which the user selects the lines, line width and color. It should be noted that the user can specify the graphic shape or not. When the user specifies the graphic shape, when generating the trajectory point data, the vertex coordinates of several vertices of the specified graphic can be generated according to the graphic shape and the image coordinate system of the writing page, and then at least one intermediate point coordinate is generated on the line between the two vertex coordinates connected in sequence. The coordinate data of each vertex coordinate and each intermediate point coordinate is the trajectory point data, and the coordinates of each vertex coordinate and each intermediate point coordinate are included in the trajectory point data.

[0257] In the case where the user does not specify a graphic shape, when generating trajectory point data, coordinate data of multiple position points can be randomly generated based on the image coordinate system of the writing page, and the order of the multiple position points can be randomly determined. The coordinate data of each position point is arranged in order to obtain trajectory point data. The following is an example of trajectory point data obtained when the coordinate data of the position points is randomly generated. The trajectory point data includes:

[0258] [(359,511,359,511),(359,511,361,518),(361,518,364,531),(364,531,369,545),(369,545,377,563),(377,563,385,577),(385,577,392,591),(392,591,398,603),(398,603,402,610),(402,610,405.7762,617.3416),(405.7762,617.3416,406.1943000000001,618.1261), (406.1943000000001,618.1261,404.2578000000001,614.7925),(416,527,416,527),(416,527,411,527),(411,527,396,535),(396,535,375,551),(375,551,362,565),(362,565,345,584),(345,584,334,596),(334,596,324,608),(324,608,318,613),(318,613,312.6793)].

[0259] Among them, each track point in the above track point data corresponds to two adjacent position points on the writing page. The data of the two adjacent position points may include their respective coordinates, such as (359, 511, 361, 518), then 359 and 511 correspond to a position point A, 361 and 518 correspond to a position point B, so that the connection relationship between each position point is determined through each track point data, that is, in the track point data of (359, 511, 361, 518), position point A and position point B need to be connected, so that the above track point data can plan the shape of a graphic. Therefore, the track point data can be drawn on the writing page according to the set line width and line shape to obtain a writing graphic.

[0260] The writing data can be understood as data on the pattern in which the writing page displays the writing track at the touch position when responding to a touch. During the test, the writing data needs to be sent to the device where the writing page is located.

[0261] Specifically, the track point data and writing data can be used as key text of element attributes, thereby generating action commands of the target data structure together with other key texts in the test information, and then mapped into operation commands. In the writing test, virtual writing or physical writing can be performed on the writing page based on the track point data. Then, the writing page can display the writing track based on the style indicated by the writing data on the writing page.

[0262] In specific implementation, when executing an operation command on a target object, writing data can be sent to a writing page, and the display component of the device where the writing page is located can be called to enable the writing page to respond to the received physical writing signal or virtual writing signal and generate a writing track on the writing page; wherein, the physical writing signal is a touch signal triggered on the writing page by a mechanical device connected to the first device based on the track point data; the virtual writing signal is a virtual touch signal sent to the writing page by the display component based on the track point data.

[0263] Virtual writing can refer to simulating a user's finger writing on a writing page when a simulated input device performs an operation corresponding to an operation command on a target object. The virtual writing is performed by the device where the writing page is located, simulating a virtual touch on the writing page based on the track point data in the operation command; while physical writing is performed by a mechanical device connected to the first device, physically touching the corresponding position on the writing page based on the track point data.

[0264] When a physical mechanical device is used, such as a robotic arm that can simulate a stylus, the touch screen can be touched according to the track points provided by the track point data. Specifically, the first device can transmit the track point data to the mechanical device, so that the mechanical device can write on the writing page according to the track point data. If the robotic arm is a six-axis robotic arm, the posture of the robotic arm can be controlled according to the track point data, so as to complete the touch of the corresponding position point on the writing page. In this process, the track point data can be converted into the posture data of the mechanical device, thereby obtaining a plurality of posture data, and then the mechanical device can change the posture according to the plurality of posture data to complete the touch of the writing page. Among them, the process of converting the track point data into posture data can refer to the conversion process between the image coordinate system and the coordinate system of the robotic arm in the relevant technology, which will not be elaborated here.

[0265] Specifically, in physical writing, since the mechanical device touches the writing page, the first device can send the track point data to the mechanical device, and send the writing data to the device where the writing page is located, so that when the mechanical device touches the writing page, the writing page can correctly respond to the touch according to the user's settings to generate a writing track of the corresponding style based on the touch of the mechanical device.

[0266] By using the embodiment of this example, the writing function of the touch display screen can be tested, and the input device can be simulated to perform virtual writing on the touch display screen, or the mechanical device can be controlled to perform physical writing on the touch display screen, thereby flexibly implementing writing tests.

[0267] As shown in Figure 4, the automatic testing architecture can include a use case layer, a parsing layer, a mapping layer, an execution layer, and a supervision layer. The supervision layer can monitor the test process and test results of the automatic test and determine the difference between the test results and the expected test results of the automatic test. In an exemplary embodiment, the test information can include assertion information, where the assertion information is used to indicate the expected test results of the target object. In other words, the assertion information is a description of the expected test results. In this way, when the processor of the first device runs the program, it can also obtain the test process information and test result information of the target object during the automatic testing process.

[0268] In a specific implementation, referring to FIG8 , a flow chart of steps for monitoring the test process is shown. As shown in FIG8 , the following steps may be included:

[0269] Step S104: Detect the test process of the target object, obtain process information, and send the process information to the communication terminal communicating with the first device.

[0270] In this embodiment, during the process of testing the target object, its test process can be detected, such as detecting the operation log of the target object in the device where it is located, so as to obtain process information. The process information can reflect the operating status of the target object when it is in the test process, which may include the above-mentioned operation log. Of course, in some other scenarios, other information may also be included, such as the support status of the device for the operation of the target object, so as to reflect the degree of fit between the target object and its device.

[0271] The process information may be sent to a communication terminal that communicates with the first device. The communication terminal may be a terminal where a tester is located or a terminal that manages a target object.

[0272] Step S105: Obtain the test result of the target object after the test is completed.

[0273] In this embodiment, after the target object is tested, its test results can be obtained; wherein, the method of obtaining the test results may be related to the type of the target object. For example, if the target object is an object with display properties, such as a space in a web page, a writing page, etc., the interface of the device where the target object is located can be called to take a screenshot of the display interface of the device where the target object is located, thereby obtaining the test results from the screenshot. For another example, if the target object is a communication element, such as a wifi element or a 5G network element on a mobile phone, the interface where the target object is located can be called to obtain the network communication data of the device where the target object is located, and the test results can be obtained through the network communication data. In other words, for target objects whose test results can be observed from the display interface, a screenshot of the display interface of the device where the target object is located can be obtained, and the test results can be obtained based on the screenshot. For target objects whose test results cannot be observed from the display interface, the interface where the target object is located can be called to obtain the status data of the target object after the test from the device where the target object is located, thereby obtaining the test results.

[0274] Step S106: Match the test result and the assertion information, and output the matching result to the display component of the first device for display, and / or send the matching result to the communication terminal.

[0275] In this embodiment, since the assertion information may include a description of the expected test results, keywords in the assertion information can be extracted, and the test results can be analyzed based on the extracted keywords. Specifically, the test results can be matched with the extracted keywords to determine whether the content included in the test results matches the keywords. If they match, it indicates that the test results meet the expected test results; if they do not match, it indicates that the test results do not meet the expected test results. Regardless of whether there is a match or a mismatch, the matching structure can be sent to the above-mentioned communication terminal. Alternatively, the results can be displayed on the first device, or both displayed on the first device and sent to the communication terminal.

[0276] When matching the test results with the extracted keywords, multiple keywords representing the test results can be first extracted from the test results. Then, the multiple keywords in the test results can be matched with the keywords in the assertion information to obtain a matching result. Specifically, if a corresponding pair of keywords does not match, the two can be considered to be mismatched.

[0277] In some examples, since the test result can be a screenshot of the display interface, the assertion information can also be an image. During matching, the similarity between the screenshot and the image can be determined. For example, if the target object is a login control in a web page, the assertion information can be the first page image that is jumped to after a successful login. After testing the login control, a screenshot of the page image after the login jump can be taken to obtain a second page image. The similarity between the first page image and the second page image can then be determined to obtain a matching result.

[0278] Of course, in some other examples, the assertion information can be text information, which can be used to describe the expected form of the test result, and the test result can be a screenshot of the display interface. Therefore, after performing image recognition on the screenshot of the display interface, multiple recognition results can be obtained, and the matching result can be obtained based on the matching between the multiple recognition results and the text information. For example, the target object is a writing page, and the writing function of the page is tested. At this time, as mentioned above, if the trajectory point data is randomly generated, it is impossible to obtain the correct page screenshot, then the assertion information can be text information, which is used to describe the correct state presented after writing, such as including a description of the shape of the graphic, a description of the line type, a description of the line width and a description of the color, etc. Therefore, after the writing test is completed, the writing page can be screenshotted, and the graphic in the screenshot image can be recognized to obtain multiple recognition results. Each recognition result can be used to describe the form of the drawn graphic in one dimension. Specifically, the recognition result can be a text type result. These recognition results can be matched with various descriptions in the assertion information to obtain a matching result.

[0279] Of course, in some other examples, both the assertion information and the test results can be text-based information. For example, if the target object is a communication component, the assertion information can be the communication status data after the test is completed, and the test result is also the communication status data obtained from the device where the target object is located. After comparing the two, a matching result can be obtained.

[0280] In this example, when the process information and test results are obtained, the process information and test results can be analyzed in multiple dimensions, so that based on the analysis results, the test personnel can better judge the operational stability of the target object. Referring to Figure 9, a schematic diagram of analyzing the process information and test results is shown. Specifically, it can include defect grading, defect identification and automatic bill of lading; wherein, defect grading can be determined based on the difference between the test results and the assertion information. The greater the difference, the higher the level of the target object's operational defect; wherein, defect identification can be determined based on the matching between the test results and the assertion information. For example, the test content in the test results that does not match the assertion information can be regarded as a defect; the automatic bill of lading can be obtained based on the process information. If there is a freeze or failure in the test process, causing the test to stop midway, the process information can be reported to the test personnel or the superior user so that the fault can be eliminated as soon as possible.

[0281] Accordingly, in some examples, defect identification can also be performed on process information, thereby identifying different defect types, such as defects such as failure to execute operation commands, abnormal page display, and non-compliant page content. When these defects occur, corresponding screenshots and logs can be retained to assist in subsequent defect repair.

[0282] In some examples, when performing defect rating, it can be determined based on the type of defect and the difference between the test results and the assertion information. For example, the greater the difference, the higher the defect rating. If the type of the defect is a specified target type, the defect rating can also be higher. The specified target type can be determined in advance by the user. For example, the specified target type can be a type of execution failure. Since it affects the normal progress of the test, the defect rating can be higher.

[0283] If, based on the test results, assertion information, and process information, a defect is determined to exist in the target object during the test process, the determined defect type and defect rating, along with the test results and process information, can be sent to the communication terminal. This process can be referred to as automatic bill of lading. In specific implementations, the defect type, defect rating, test results, and process information can be encapsulated using HTTP via the interface between the first device and the communication terminal. The encapsulated data packet can then be sent to the communication terminal via the HTTP protocol. Testers at the communication terminal can then repair the fault based on the defect type, test results, and process information provided in the data packet. For example, they can repair a library file accordingly to eliminate the fault. Specifically, after the library file is repaired, the repaired library file can be returned to the first device. Subsequently, the first device can map the test information containing the defect to an action sequence based on the new library file and test the target object. If the test result is successful, the success result can be reported to the communication terminal. If the test result is still unsuccessful, the defect reporting process can be continued to report to the communication terminal.

[0284] In the process of acquiring process information, different aspects of the test process can be monitored for different target objects. For example, when applied to a web page test scenario, it can automatically monitor resources, page UI (User Interface) layout, page display, device status, and exception logs.

[0285] In some examples, when detecting the target object during the test, at least one of the script data of the current web page, the display interface of the current web page, the status information of the current web page, and the operation log of the current web page can be monitored.

[0286] Among them, the script data can refer to the UI data of the current web page, thereby determining whether the elements in the web page will affect the UI layout of the web page during the testing process; wherein, the detection of the display interface can determine whether the display interface has display abnormalities such as black screen or flowery screen during the testing of the elements in the web page; wherein, the status information of the web page can be used to characterize the display status of the web page, such as whether garbled characters are presented, whether the display is clear, etc.; wherein, the operation log can be used to reflect whether the web page has abnormalities, and the abnormal information of the web page during the display process can be monitored from the operation log.

[0287] In some other examples, regardless of whether the target object to be tested is a terminal application or a terminal device, at least one of the following information can be monitored during the test: processor usage, memory usage, frame rate of the current web page, data traffic of the current web page, power consumption of the device where the target object is located, usage of the graphics processor of the device where the target object is located, and network connection status of the device where the target object is located.

[0288] The processor can refer to the CPU of the device where the target object is located, and the processor usage rate can refer to the CPU usage rate. During the target object testing process, the CPU usage rate before and after the test can be used to determine the CPU usage of the target object and the operating environment provided by the CPU when the target object is running. Similarly, the memory usage rate can also reflect the memory usage of the target object and the hardware support performance provided by the memory for the target object's operation. The frame rate of the current web page can be used to reflect the frame rate of the page where the target object is located and the impact of the target object's operation on the frame rate of the web page, thereby indirectly reflecting whether the web page has display abnormalities. The data flow rate can be the data support provided by the web page during the operation of the target object, and can reflect the impact of the target object's operation on the data flow of the web page, that is, whether the target object's operation affects the acquisition of web page data. It can also be used to represent the acquisition status of the web page data of the target object after the test. The power consumption of the device where the target object is located can reflect the power consumption corresponding to the operation of the target object. The graphics processor usage rate, like the CPU usage rate, can reflect the graphics processor usage of the target object. The network connection status can be used to represent the network status of the target object during operation.

[0289] Using the above example, the software and hardware performance parameters of the device where the target object is located can be obtained during the test of the target object, so that the software and hardware environment in which the target object is running, as well as the impact of the target object's operation on the software and hardware performance, can be clarified. This allows for more comprehensive monitoring of the target object's test process and acquisition of more accurate and comprehensive process information.

[0290] As shown in Figure 1, a writing test scenario can also be included. In this test scenario, the target object can be a writing page, and the device where the target object is located has a touch screen display. The touch screen display can serve as a hardware device for providing writing on the writing page. In this case, if it is necessary to monitor the test process and test results, when matching the test results with the assertion information, the completed writing trajectory can be image recognized, and then matched with the assertion information based on the recognition result.

[0291] In specific implementation, when obtaining the test results of the target object after the test is completed, the interface of the device where the writing page is located can be called to obtain the display interface image after the writing test is completed, and at least one of the following contents of the writing track in the display interface image can be identified: font, line width, color, graphic shape and pen shape; then, the test results can be obtained based on the recognition results and assertion information.

[0292] In this embodiment, the first test information is information for testing the writing function of the writing page, wherein, in the writing function test of the writing page, the assertion information may include a morphological description of the graphics after writing, and the test result obtained may be a display interface image obtained by taking a screenshot of the display interface of the writing page. The display interface image may include a writing track drawn by a device where the writing page is located after performing virtual writing according to the track point data, or may be a writing track obtained by a mechanical device performing physical writing on the writing page according to the track point data, wherein the test result may be obtained by identifying each content in the writing track. Specifically, at least one of the font, line width, color, graphic shape and pen shape in the writing track may be identified to obtain a recognition result. It should be noted that the recognition result may be a text type result, and thus may belong to the same format type as the morphological description of the writing track in the assertion information, and may be subsequently compared.

[0293] In one example, the entire content of the writing track of the display interface image, including the font, line width, color, graphic shape, and pen shape, can be identified to obtain morphological features of various aspects of the writing track. When matching with the assertion information, a target morphological feature of the same morphological type as that described in the assertion information can be selected from the various morphological features identified in the writing track, and then the target morphological feature can be compared with the morphology described in the assertion information to obtain a test result. For example, the assertion information includes an expected width of the line width and an expected shape of the line shape. Therefore, the recognition results of the line width and the recognition results of the line shape in the writing track can be compared with the expected width of the line width and the expected shape of the line shape, respectively, to obtain a test result.

[0294] In another example, when identifying a writing track in a display interface image, the content to be identified can be determined according to the type of morphological description included in the assertion information, and then the content to be identified in the writing track can be identified. In other words, based on the assertion information, it can be determined which content in the writing track needs to be compared with the assertion information. For example, if the assertion information includes a morphological description of line width and a morphological description of font, the line width and font style can be identified from the writing track, and the identified line width and font style can be compared with the morphological description of line width and the morphological description of font in the assertion information, respectively. In this way, the types of content to be identified in the display interface image can be reduced, thereby improving the efficiency of determining the test results.

[0295] Among them, when identifying the writing track in the display interface image, the display interface image can be input into a pre-trained writing model, and the various contents in the writing track can be identified through the writing model. The writing model can extract features from the input display interface image, and then, based on the extracted features, the line width, line shape and color of the lines in the writing track, the shape of the written graphics, the written font style, etc. can be determined to output the recognition result. Of course, when identifying the writing track, if the features of the same type as the morphological type described by the assertion information in the writing track are to be identified, then the display interface image is input into several specific branches in the writing model, so that each specific branch can be used to identify a type of feature in the writing track and output the recognition result. In this example, the writing model can include multiple branches, and different branches can be used to identify different contents in the writing track.

[0296] Among them, the training process of the writing model can be as follows: prepare multiple writing track image samples, and the correct recognition results of each content in each writing track image sample, and use the correct recognition results as labels for training supervision. Then, input the multiple writing track image samples into the preset model. The preset model may include a feature extraction unit, and multiple branches connected to the feature extraction unit. The feature extraction unit is used to extract features of the writing track image samples, and then the feature extraction results are input into multiple branches respectively. Each branch can analyze the input feature map to output the recognition result corresponding to the branch; then, the recognition result output by each branch is compared with the correct recognition result to calculate the loss function, and the parameters of the preset model are updated according to the loss value. The parameters of the preset model can include the parameters of the feature extraction unit and the parameters of each of the multiple branches.

[0297] Among them, since in the writing test, the line width, color, shape, etc. of the writing track obtained after the test is completed can be identified, and the identification result can be judged in combination with the assertion information to determine whether the writing track meets the expected test results, thereby realizing automated writing testing and automated writing test result judgment, without the need for users to manually judge the test results, thereby improving the automation performance of the first device during the test process and optimizing the user experience.

[0298] In some examples, a target object may have multiple test items. For example, in a login test, the test items include a username validity check and a password verification test item. The username validity check item may further include a null value check, a length check, and a letter and character check. In practice, test information needs to be designed for each verification-related test item in the login item. In some embodiments of this example, based on a basic test information, the test information required for multiple test items included in the test item to which the test information belongs can be automatically generated, thereby improving the efficiency of designing test information for multiple test items included in the same test item.

[0299] In specific implementation, a use case database can also be stored in the memory of the first device, which includes multiple test types and multiple preset test parameters corresponding to each test type, where different preset test parameters correspond to different test items under the same test type.

[0300] When a piece of test information is obtained, multiple target preset test parameters matching the target test type carried in the test information can be retrieved from the use case database. Based on each target preset test parameter, multiple pieces of basic test information associated with the test information can be generated. Subsequently, the multiple pieces of basic test information can be sent to a use case source connected to the first device. Different pieces of basic test information are used to implement different test items under the test information.

[0301] In this embodiment, the test type carried in the test information can be located in a preset field in the test information. The test type can represent the type of test that the test information is to perform on the target object. In some cases, it can also ensure the type of function implemented by the target object to which the test information belongs. For example, if the target object is a writing page, the test type can be a writing test. If the target object is a login control in a web page, the test type can be a login type.

[0302] Among them, when the first device obtains the test information, it can determine the target test type to which the test information belongs from the test information, and then, it can obtain multiple target preset test parameters matching the target test type from the use case database. The multiple target preset test parameters can be understood as test parameters for different test items under the same test project. Then, based on the multiple target preset test parameters, multiple basic test information can be generated, that is, each target preset test parameter can generate a basic test information, and the generated multiple basic test information can be used to complete the test of different test items under the test information.

[0303] Specifically, each preset test parameter may include input parameters to be used by different test items, such as input parameters of element attributes, which can be understood as the contribution made in the test to input configuration parameters that the test depends on into the target object. For example, if the target object is a login control, the preset test parameter may be the user name and password entered under the corresponding test item. Of course, in some other examples, the preset test parameters may also include corresponding operation action parameters, which may be used to indicate the action to be performed by the input device in the test. The operation action parameter may be a parameter of an action attribute.

[0304] Below, a specific example is given for illustration. In this example, the target object is a login control in a web page, and the test information input to the first device is information of the login test type. The login test may include test items for username and password verification, which can be specifically divided into null value verification, username length verification, and username content verification. Among them, the null value verification, username length verification, and username content verification each include verification tests in multiple situations. Therefore, the verification test in each situation can be called a test item; specifically, the first device can obtain multiple target preset parameters under the login test from the use case source. The multiple target preset parameters may include multiple preset parameters belonging to null value verification, multiple preset parameters belonging to username length verification, and multiple preset parameters belonging to username content verification; different preset parameters among the multiple preset parameters belonging to null value verification correspond to different test items under the null value verification.

[0305] As shown in Table 1 below, Table 1 shows various target preset parameters under the login test:

[0306] Among them, multiple preset parameters belonging to the empty value check may include the preset parameters of the username being empty, the password being not empty, and clicking to log in, the preset parameters of the username being not empty, the password being empty, and clicking to log in, and the preset parameters of the username and password being not empty, and clicking to log in; similarly, as shown in Table 1, 5 preset parameters can be included under the username length check, and 3 preset parameters can be included in the username content check.

[0307] Different preset parameters can generate a basic test message. It can be seen that each preset parameter can include parameters of element attributes and action attributes. This makes the preset parameters constitute the elements required for test information, so that a new basic test message can be automatically generated. In this way, users no longer need to edit the corresponding test information for null value verification, username length verification, and username content verification.

[0308] After generating the basic test information, the target object can be tested according to the basic test information. The testing process can be similar to the testing process in the test information, which is not described in detail here. Furthermore, after generating the basic test information, the basic test information can be sent to the use case source for storage. Specifically, the basic test information can be sent to the use case source from which the test information originated, as a use case supplement to the test information.

[0309] By adopting this embodiment, the test efficiency of automatic testing can be greatly improved. For example, in the login test, in the username input box on the login page, the user needs to check whether it is empty, normal / abnormal length, normal / abnormal content, and it may be necessary to write 11 use cases to fill in all of them; after adopting the solution of this embodiment, the user can abbreviate: username input box check [length: 5~10, content: letters|numbers], so that the first device can automatically generate supplementary basic test information based on the abbreviated test information, which can greatly improve the user's test case design efficiency.

[0310] In some examples, the implementation of a test project may require other test projects to be carried out on the target object as a prerequisite. For example, the use case numbered GN-01-01 is the successful login of the user; the use case numbered GN-01-02 is the creation of a work order. Then, the execution of the use case GN-01-02 requires a prerequisite step GN-01-01 (i.e., login is required before creating a work order).

[0311] The test information may include a note about pre-test items that need to be performed before the test item is performed. Specifically, the test information may include a target field, which may be used to indicate the second test information that the current test information depends on. When the program is executed by the processor, it is also necessary to obtain the second test information based on the content of the target field, and test the target object based on the second test information before the current test information can be tested. In this way, the second test information is equivalent to an operation process in the current test information. Specifically, the second test information can be obtained from a use case source that communicates with the first device in response to the content of the target field. The test indicated by the second test information is performed on the target object, and after the test is completed, the test indicated by the test information is performed on the target object.

[0312] In this embodiment, after the first device obtains the test information, if it is determined through the target field of the test information that it has the second test information of the pre-test, the second test information can be obtained first. Specifically, the second test information can be queried from multiple use case sources connected to the first device, and then the test performed by the second test information is executed, wherein the second test information and the test information received by the first device can be for the same target object or for different target objects, wherein the execution process of the second test information can refer to the execution process of the test information by the first device mentioned above, such as first parsing multiple key texts from the second test information, then combining the multiple key texts into an action sequence of the target data result, and then mapping the multiple action sequences into operation commands based on multiple library files, and executing the operation commands on the target object targeted by the second test information.

[0313] After the second test information is executed, the currently received test information may be executed to test the target object.

[0314] For example, assuming that the test information received by the first device is information for creating a work order, which indicates the test information of the login test as the preceding second test information, the first device first obtains the second test information of the login test, enters the user name and password in the login control through the second test information, and jumps to the target interface after login. Then, the first device begins to parse, combine and map the test information, and obtains the operation command corresponding to the work order creation element on the target interface. After executing the operation command, the work order created by the test work order creation element can be obtained, and the work order is the test result.

[0315] The first device and the automatic test performed by the first device are described below through two exemplary embodiments:

[0316] Example A:

[0317] Referring to Figures 10, 11a and 11b, Figure 10 shows a schematic diagram of the communication environment architecture where the first device is located, and Figures 11a and 11b show a flowchart taking the target object as a login control in a web page as an example. As shown in Figure 10, it includes a first device, and an AI platform, a test case management platform and a test case database connected to the first device, wherein the test case management platform and the test case database are used to provide test information for the first device, and the AI ​​platform is used to provide algorithm support for the first device, such as various neural network models provided in the positioning of the target object, to locate the target object based on image positioning; wherein the CI / CD service can be a Jenkins service, or a continuous integration service, which completes the automatic execution of use cases according to code and policy settings.

[0318] It may include the following processes:

[0319] The main implementation process is as follows:

[0320] 11) Initialize the program and various library files in the memory, select the test information path to be parsed and executed, the device where the login control is located, set the log level and object identification execution mode, and thus complete the automatic test environment construction;

[0321] 12) Process the test information

[0322] First, read the test information from the use case source. The test information can be the test information of the table class, as shown in Table 2 below:

[0323] In Table 2, module represents the module to which the current test information belongs. The scope can be large or small, and a page or functional block is generally considered a module. Test item refers to the concept of a submodule. Test type refers to the test type of the test information, such as functional test, reliability test, efficiency test, etc., which is slightly different from the test type in the above embodiment. The above test type refers to the category corresponding to the target object and is mainly used when generating basic test information. Generally speaking, test information includes not only the test types in Table 2, but also the test types used to generate basic test information. These test types and the test types in Table 2 can be in different fields of the test information. Priority represents the priority of the test information. The use case number is used to uniquely identify the test information. Precondition is the precondition used as the test input, that is, the target field mentioned in the above embodiment, in which the identifier of the second test information can be identified. Expected output is the assertion information mentioned above, which serves as the truth assertion of the test output. Test input is the key part of the test information. Generally, the key text is parsed from the test input. Table 2 shows three types of test input, where each test input corresponds to an independent test information. In other words, a row of data in Table 2 represents one test information.

[0324] The test information of the type in Table 2 above can be understood as a template of test information. When the user edits the test information, he or she can fill in the specific information in the template.

[0325] Then follow these steps in sequence:

[0326] a) Read each column in Table 2 and use the test number as the unique identifier of the test information; determine whether the tag should skip this test information (the row where the column is located);

[0327] b) If there is no need to skip, the precondition, test input, and test output of the test information in this row are further parsed. If the precondition requires testing the precondition second test information first, the second test information is read from the use case source, and the target object corresponding to the second test information is tested. After the test is completed, the test result of the second test information is saved on the device where the target object is located. Then, based on the test result, the test of the test information in this row is completed, and the process proceeds to step c). If the precondition does not require testing the second test information, the process proceeds directly to step c):

[0328] Table 2 Information distribution table included in the test information

[0329] c) Standardizing the test information, including parsing multiple key texts from the test information and designing the operation commands in the form of verb + noun + [parameters]. Specifically, the test information, i.e., the aforementioned test input, can be first split into multiple paragraphs using Chinese word segmentation. Then, based on the keyword dictionary whitelist and blacklist, multiple phrases with parts of speech are decomposed from the multiple paragraphs. The multiple phrases are converted into action sequences that conform to the target data structure of verb + noun + [parameters] according to their parts of speech;

[0330] Afterwards, the key text of the action attribute in the action sequence (action + object + parameter) is mapped to the operation keyword in the action library file to obtain the operation-related command line; the key text of the object attribute in the action sequence (action + object + parameter) is mapped to the object library file to obtain the key value (position) of the returned target object; the key text of the element attribute in the action sequence (action + object + parameter) is mapped to the test database file to obtain the returned element-related command line, and then the mapped command lines are combined in the form of "verb + noun + [parameter]" to obtain the operation command.

[0331] Among them, in the process of mapping the key text of the object attribute in the action sequence (action + object + parameter) to the object library file to obtain the key value (position) of the returned target object, the object library file stores the names of multiple elements in the web page, and the description information of the target object included in the test information is "login". Therefore, you can first search for the element matching "login" in the object library file, and return the position data of the element in the web page as the key value.

[0332] 13) Execute operation commands

[0333] The operation command is sent to the device where the web page is located, such as the second device, so that the web terminal of the second device can call the action library based on the Selenium secondary encapsulation to perform operations such as clicks and input to complete the test. For example, the input device can be simulated to perform the operation corresponding to the operation command on the login control in the web page. For example, the mouse is first simulated to locate the login control in the web page according to the key value in the operation command, then the keyboard is simulated to enter the corresponding username and password in the username and password fields, and then the mouse is simulated to click on the login control to obtain the login interface.

[0334] 14) Inspection of test process and test results

[0335] Among them, for the monitoring of the test process, the second device can be used for resource monitoring, web page UI layout monitoring, page display monitoring, equipment status monitoring, and abnormal log monitoring; at the same time, the interface return can be monitored and the interface return results can be accurately / fuzzy matched; and the test path recording can be completed at the same time.

[0336] The resource monitoring may monitor the resource usage of the second device, such as CPU / memory / fps / data flow / power / GPU usage;

[0337] Among them, UI layout monitoring can detect whether there are any abnormalities in the syntax of the page source code [HTML, CSS, JAVASCRIPT];

[0338] Among them, display monitoring can detect image UI displays (such as empty values, text overlap, and cracked images), compare the differences between the page UI and the baseline UI, and accurately locate the differences;

[0339] Among them, device status monitoring can monitor the connection status of the terminal device. If the device has a connection abnormality, it will try to reconnect automatically ("adb connect");

[0340] Among them, abnormal log monitoring can monitor the logs of crash / oom on the Android side;

[0341] Among them, interface response monitoring can monitor page interface returns and accurately / fuzzily match interface return results (return status code / return content, etc.).

[0342] In this case, a screenshot of the page that is redirected after clicking the login control can be taken, and the captured image can be analyzed and identified to obtain the test results. For example, the screenshot image can be monitored for black screen, distorted screen, and garbled characters to determine whether the redirected page is the page expected in the assertion information.

[0343] 15) Test result synchronization

[0344] Based on the process information of the test process, it is determined whether there is an abnormality in the test process. If an abnormality occurs, it is fed back to the communication terminal to inform the developer. The developer can repair the defect based on the code in the code repository, and the repaired code can be fed back to the first device, such as feeding the repaired library file back to the first device for storage; if there is no abnormality, it is determined whether there is an abnormality in the test result. Specifically, the test result is matched with the assertion information. If an abnormality occurs, it is fed back to the communication terminal to complete the process of automatic bill of lading.

[0345] As shown in FIG11 , during the synchronization of test results, defect identification, defect grading, and automatic billing can be achieved based on process information and test results. Then, based on the results of defect identification, developers can perform defect repairs, and the results of defect repairs can be fed back to the first device, so that the first device can apply the defect repairs to use case analysis and mapping.

[0346] Example B:

[0347] Referring to Figures 12 and 13, taking the target object as a writing page as an example, Figure 12 shows a schematic diagram of the environmental architecture where the first device is located, and Figure 13 shows a flow chart of the writing test. As shown in Figure 12, it includes a first device, and an AI platform, a test case management platform, and a test case database connected to the first device, wherein the test case management platform and the test case database are used to provide test information for the first device, and the AI ​​platform is used to provide algorithm support for the first device, such as various neural network models provided in the positioning of the target object, so as to locate the target object based on image positioning. Among them, the first device can also be configured with a robotic arm and an infrared simulator, wherein the robotic arm and the infrared simulator can be used for physical writing. As shown in Figure 13, it may include the following processes:

[0348] 21) As in step 11), initialize the program and various library files in the memory, select the test information path to be parsed and executed, the device where the writing page is located, set the log level and object identification execution mode, and thus complete the automatic test environment setup;

[0349] 22) Process the test information

[0350] First, test information is read from the use case source. The test information may be table-type test information, as specifically shown in Table 2 above. The difference is that, in a written test case, the test input may include a "writing" tag, thereby prompting the first device to first pop up a writing setting page so that the user can set the writing in the writing setting page.

[0351] 23) Analysis of test information

[0352] As shown in step 12) above, it is still necessary to read each column to determine whether the second test information needs to be executed first. If not, the analysis of the current test information is entered. The specific process is as follows:

[0353] d) Displaying a built-in writing setting page, which presets writing templates such as text, numbers, letters, formulas, graphics, and lines; in response to the user's settings in the writing setting page, based on the selected writing template, a handwriting learning generation algorithm is used to generate track point data and writing data. The writing data is generated according to content and style, and is used as a writing test data source, which can also be understood as key text of element attributes in the test information;

[0354] e) Standardize the test information and design the operation commands in the form of verb + noun + [parameter]. Specifically, based on the keyword dictionary whitelist and blacklist, the action attribute phrases can be extracted from the test information of the writing test, with the writing data and trajectory point data as the element attribute phrases and the writing page as the object writing phrase, and converted into an action sequence that conforms to the target data structure of verb + noun + [parameter].

[0355] f) Map the key text of the action attributes in the action sequence (action + object + parameter) to the operation keywords in the action library file to obtain the command line related to the operation; map the key text of the object attributes in the action sequence (action + object + parameter) to the object library file to obtain the key value (position) of the returned writing page; combine the trajectory point data, writing data and the above command line in the form of "verb + noun + [parameter]" to obtain the operation command.

[0356] 23) Execute operation commands

[0357] The trajectory point data in the operation command is sent to the robotic arm, and the other data in the operation command is sent to the writing page. The first device maintains socket communication with the robotic arm. The robotic arm converts the trajectory point data into posture data for controlling its own posture, so as to perform touch on the touch display screen of the second device. The writing page responds to the touch position and generates corresponding lines according to the writing data, thereby obtaining a writing trajectory.

[0358] 24) Inspection of test process and test results

[0359] Among them, for monitoring the test process, the second device can be used for resource monitoring, writing page display monitoring, equipment status monitoring, and abnormal log monitoring; at the same time, the interface return can be monitored and the interface return results can be accurately / fuzzy matched; and the test path recording can be completed at the same time.

[0360] The resource monitoring may monitor the resource usage of the second device, such as CPU / memory / fps / data flow / power / GPU usage;

[0361] Among them, display monitoring can detect image display, such as: empty value / text overlap / cracked image, distorted screen, black screen, etc.

[0362] Among them, device status monitoring can monitor the connection status of the terminal device. If the device has a connection abnormality, it will try to reconnect automatically ("adb connect");

[0363] Among them, abnormal log monitoring can monitor the logs of crash / oom on the Android side;

[0364] Among them, interface response monitoring can monitor page interface returns and accurately / fuzzily match interface return results (return status code / return content, etc.).

[0365] Among them, the display interface image of the writing page can be obtained, so as to analyze and identify the writing track in the display interface image to obtain the test results, which include the recognition of the line width, color, shape, etc. in the writing track, and the recognized line width, color, and shape are respectively compared with the line width, color, and shape in the assertion information, so as to complete the assertion based on the comparison results.

[0366] Based on the above display monitoring, the screen quality is obtained, such as whether there is a distorted screen, a black screen, and whether there are abnormal text displays, image content, video content, etc.

[0367] The embodiment of the first device has the following advantages:

[0368] First, it can automatically parse customized test information and convert it into a standardized action sequence, thereby obtaining operation commands that can be executed by the device where the target object is located, so as to automatically test the target object. There is no need for testers to write test scripts in advance, which greatly reduces the requirements for testers and labor costs. In addition, compared with writing test scripts, the time spent on automatic parsing is greatly shortened, thereby improving the test efficiency of automatic testing.

[0369] Second, based on the three major libraries (test database file, action library file, and object library file), multiple methods can be flexibly used to map to operation commands. For example, in object positioning, image recognition, OCR content recognition, and attribute information based on elements in the page can be used, allowing test information to be written more freely and optimizing the test experience of testers.

[0370] Third, when locating the target object, it can be based on the feature maps of each element in the web page. In this way, when the UI design of the web page is completed, a fixed set of feature maps can be formed, which can be applied to the test of the same element on multiple terminal devices. In other words, it can be suitable for multi-terminal object feature matching and has strong scalability.

[0371] In general, by using the first device of the present application, the test information design can be directly executed without the need for test script writing, thereby improving the convenience, intelligence and reviewability of the test process.

[0372] Based on the same inventive concept, the present disclosure also proposes a second device that can, in response to an interface call and operation command from the first device, operate a target object on the second device to complete the automatic test instructed by the first device. In this embodiment, the second device and the first device can be two independent devices that can be connected via communication, such as socket communication.

[0373] In conjunction with FIG2 , in a user scenario where a second device is located, the second device is configured with a target object and a display component, as well as a program that supports the operation of the target object. When the program is executed by the processor of the second device, it is used to perform an automatic test instructed by the first device connected to the second device. FIG14 is a flow chart showing the steps performed by the second device in the automatic test. As shown in FIG14 , the automatic test performed by the second device includes:

[0374] Step S201: receiving an operation command sent by the first device through an interface;

[0375] Step S202: performing an operation corresponding to the operation command on the target object;

[0376] Step S203: Displaying the screen after executing the operation command on the display component;

[0377] The operation command is obtained by mapping the standardized action sequence of the test information to the action sequence when the processor in the first device runs the program. The action sequence represents the operation process performed on the target object when executing the test content.

[0378] In this embodiment, the second device is configured with a target object to be tested. The target object can be an element in a web page or an independent terminal application, such as a writing application. The operation command is an action sequence obtained by mapping the first device after parsing the test information. The action sequence is obtained by combining multiple key texts in the test information according to the target data structure. Specifically, the first device can first parse the test information to obtain multiple key texts related to the test, and then construct the multiple key texts into an action sequence of the target data structure.

[0379] Among them, the key text included in the action sequence may include the text of object attributes, the text of element attributes and the text of action attributes. The text of object attributes is used to identify the target object, the text of element attributes is used to indicate the parameters input to the target object, and the text of action attributes is used to represent the operation actions performed on the target object.

[0380] Among them, after receiving the operation command, the second device can simulate the input device to perform the operation corresponding to the operation command on the target object. For example, the second device can map the operation command to the command framework when the device runs the target object, thereby performing the operation on the target object on the second device, and then displaying the screen after performing the corresponding operation on the target object.

[0381] The specific process of converting the test information into the operation command can refer to the description of the relevant parts of the embodiment of the first device mentioned above, and will not be repeated here.

[0382] When this embodiment is used, when the target object running on the second device responds to the operation command sent by the first device, it can display the entire process of responding to the operation command. If the target object is an element in a web page, such as a login control, the entire process of executing the login control, such as entering a username and password, and clicking on the element, can be displayed on the web page. The page to which the login control jumps after clicking login can also be displayed. Thus, by changing the display interface on the second device, it can be determined whether the test of the target object is successful. Therefore, when it is necessary to test the performance of the target object running on the second device, there is no need for the tester to write a test script that can be run by the second device. Instead, the first device processes the customized test information, making it possible to test the target object on the second device. The tester can determine whether the test process is smooth and whether the test results meet expectations by watching the changes in the display interface of the second device. This can optimize the tester's testing experience.

[0383] As shown in Figure 1, the automatic test can also include a test scenario of the terminal device. In this test scenario, the target object can be a writing page, and the writing function of the second device can be tested. Then, in the second device, when an operation command arrives, it can use virtual writing or physical writing to write on the writing page, wherein the physical writing can be performed based on the touch of a mechanical device. Specifically, in virtual writing, based on the trajectory point data, the user's finger can be simulated to perform a virtual touch on the touch screen, and based on the writing data, a writing trajectory corresponding to the position of the virtual touch is generated on the writing page;

[0384] Alternatively, based on the operation command, in response to a touch on the touch screen by a mechanical device connected to the first device, a writing track corresponding to the touch position is generated on the writing page based on the writing data;

[0385] The mechanical device is used to perform touch control on the touch display screen based on the track point data sent by the first device.

[0386] In this embodiment, in a writing test scenario, the writing function of the writing page can be tested. Specifically, the second device can be configured with a touch screen, which is used to receive touch. In the writing test, touch can include virtual touch and physical touch. Virtual touch can refer to giving a virtual touch electrical signal to the corresponding position of the touch screen. For example, virtual touch data is sent to the touch chip so that the touch chip believes that a touch has occurred at the real touch position.

[0387] The operation command may include track point data and writing data. The track point data is used to indicate the touch location on the touch screen, and the writing data is used to indicate the form of the graphic formed by connecting multiple touch locations, such as color, line width, line shape, and font shape. Accordingly, in one embodiment, the second device can perform virtual writing. In virtual writing, a virtual touch electrical signal can be sent to the corresponding position of the touch screen based on the track point data, such as sending virtual touch data to the touch chip, so that the writing page responds to the touch. Based on the writing data, a corresponding writing trajectory is generated according to each touch location point.

[0388] In another embodiment, the touch signal received by the touch screen can be a real touch signal. In this embodiment, a mechanical device can be used to touch the touch screen, such as a robotic arm. The trajectory point data can be sent to the mechanical device and converted into posture data by the mechanical device, so that touch can be performed at various positions on the touch screen. The second device can respond to the real touch position and generate a corresponding writing trajectory according to the style indicated by the writing data.

[0389] Therefore, in this embodiment, the second device can accept the control of the external first device and test its own writing function.

[0390] Among them, the test scenario of the terminal device can also include the response stability of the terminal in response to an external remote control device, such as the response stability of the terminal device under frequent operation of an external infrared remote control signal; for example, the power supply of the electronic device will be frequently cut off to determine the electrical stability of the terminal device; for example, the network connection of the electronic device will be frequently cut off to test the network response stability of the terminal device. Then, when the second device is used as the object to be tested, the various hardware included in it can also be automatically tested through the test information. Then, when the second device responds to the operation command of the first device to operate, at least one of the following operations can be performed: simulate the operation device to continuously touch the page button of the web page to change one of the following on the displayed page: display brightness, page open state and page size; simulate the operation device to continuously touch the switch of the communication button to change the network connection state of the second device; simulate the operation device to switch the signal source from which the displayed page comes to jump from one signal source to another.

[0391] In this embodiment, the operating device can be an infrared remote control or a keyboard or mouse connected to the second device. When the switch of the page button is continuously touched, any one of the display brightness, page opening state and page size of the displayed page can be adjusted. Specifically, in the adjustment of the page size, its elements can be the maximized element, minimized element and close control of the page. Frequent adjustment means that the switch is touched at least N times within a preset time, and the preset time can be one minute or one second.

[0392] The communication button can include a Wi-Fi switch, a 5G signal switch, a GPS signal switch, and the like on the second device. Continuously touching the communication button can frequently turn the communication connection of the second device on and off, such as frequently turning the Wi-Fi signal, 5G signal, and GPS signal on and off. This test can be used to test the stability of the second device under frequent changes in network connection.

[0393] Among them, the signal source connected to the second device can also be switched. Generally, the signal source can provide a display page, so that the second device frequently jumps among the display pages provided by multiple signal sources. For example, if the second device is a TV, the frequent switching of display pages can be the frequent switching of TV programs (signal sources). In this way, the stability of the second device when the display pages are frequently switched can be tested.

[0394] For example, the operating device may be an infrared remote controller. When the second device receives the operating command corresponding to the test information, it may simulate the key value of the infrared remote controller to perform any of the following tests on the second device:

[0395] 1) Turn the device on and off 200 times: Turn off the device -> Turn on the device -> Wait for the device to start up -> Take a photo -> Turn off the device -> Wait for the device to shut down.

[0396] 2) WiFi stability after 100 consecutive on / off cycles.

[0397] 3) Switching between different signal source channels.

[0398] The second device using this embodiment can accept the operation command of the first device and test the performance of the hardware and software configured in the second device in response to the operation command.

[0399] Based on the same inventive concept, the present disclosure adaptively proposes a test system, as shown in FIG15 , which shows a user scenario diagram of the test system. As shown in FIG15 , the test system may include a first device and a second device, wherein the first device is configured with a first page, and the second device is configured with a target object and a second page. The second page may be a page presented by a display component of the second device. Specifically,

[0400] The first device may be configured to, in response to an automatic test triggered on the first page, obtain test information, convert the test information into a standardized action sequence, and map the action sequence into an operation command adapted to the target object; wherein the test information is used to describe the test items to be performed on the target object, and the action sequence is used to represent the operation process performed on the target object when executing the test content;

[0401] The second device may be configured to execute an operation command on the target object in response to the first device calling the interface where the target object is located, and to display the execution process and results of the operation command on the second page.

[0402] In this embodiment, the first device and the second device can be communicatively connected, wherein the process of the first device generating an operation command can refer to the description in the above-mentioned embodiment of the first device, and the process of the second device responding to the operation command and performing a corresponding operation on the target object can also refer to the description in the above-mentioned embodiment of the first device and the description in the embodiment of the second device, which will not be repeated here.

[0403] For example, the examples shown in FIG10 and FIG11 are used to illustrate how the test system performs automatic testing. Referring to FIG16 , the following process may be specifically included:

[0404] 31) The first device initializes the program and various library files in the memory, selects the test information path to be parsed and executed, the device where the login control is located, sets the log level and object identification execution mode, thereby completing the automatic test environment construction;

[0405] 32) The first device processes the test information, such as the test information shown in Table 2, which requires a login test to be performed on a login control in a web page on the second device;

[0406] Wherein, the operation command can be obtained according to step 12);

[0407] 33) The first device sends the operation command to the second device;

[0408] 34) The web terminal of the second device can simulate input devices based on the Selenium secondary encapsulated action library, calling the action library to perform clicks, inputs, and other operations to complete the test. For example, it can simulate the input device to perform the operation corresponding to the operation command on the login control in the web page, such as first simulating the mouse to locate the login control in the web page according to the key value in the operation command, then simulating the keyboard to enter the corresponding username and password in the username and password fields, and then simulating the mouse to click the login control to obtain the login interface; as shown in Figure 16, the login interface includes work order entry, work order information, and executor information;

[0409] 35) The second device takes a screenshot of the interface after login and detects the test process to obtain process information. The second device feeds the screenshot and process information back to the first device;

[0410] 36) The first device determines whether there is any abnormality in the test process based on the process information of the test process. If there is any abnormality, it feeds back to the communication terminal; if there is no abnormality, it determines whether there is any abnormality in the test result. Specifically, it matches the test result with the assertion information. If there is any abnormality, it feeds back to the communication terminal to complete the automatic bill of lading process.

[0411] Based on the same inventive concept, the present disclosure adaptively proposes a testing method, which can be applied to the cloud, to the first device described above, or to the second device described above. Specifically, referring to FIG. 17 , a flowchart of the steps of the testing method is shown. As shown in FIG. 17 , the testing method can include the following steps:

[0412] Step S301: Acquire test information to be executed; wherein the test information is used to describe the test items to be performed on the target object;

[0413] Step S302: parsing the test information to extract multiple key texts related to the test, and constructing the multiple key texts into an action sequence of the target data structure;

[0414] Among them, the action sequence is used to represent the operation process performed on the target object when executing the test content;

[0415] Step S303: Mapping the action sequence into an operation command adapted to the target object, and executing the operation command on the target object to complete the test indicated by the test information on the target object.

[0416] In this embodiment, the test information to be executed can be user-defined test information, specifically text information, table information, or image information, or the test information can include both text information and image information, or both table information and image information. Generally speaking, a test information tests a target object, which can be a specific application or a hardware component on a device. For example, the target object can be an element on a web page, such as an element, or it can be a touch screen, power switch, communication switch, etc. on the device.

[0417] Whether the test information is textual, graphical, or graphical, it can include description information, test type, and configuration information. The description information describes the target object, the test type indicates the type of test to be performed on the target object, and the configuration information specifies the parameters to be input to the target object during the test. In practice, the test information can be parsed to obtain the key text in the description information, test type, and configuration information.

[0418] Among them, during parsing, if the test information is of text type, the key text can be extracted from the test information. If it is of table type, the text information can be extracted from the table type test information first, and then the key text can be extracted from the text information. If it is of image type test information, the text in the image can be recognized first, and then the key text can be extracted from the recognized text.

[0419] After parsing the test information, the multiple key texts obtained can be divided into key texts describing the target object, key texts describing the input, and key texts describing the operation action. For example, the key texts can be divided into key texts describing action attributes, key texts describing element attributes, and key texts describing object attributes based on their part of speech. The multiple key texts parsed are all closely related to the test of the target object. After obtaining these multiple key texts, they can be constructed into an action sequence that conforms to the "verb + noun + parameter (element)" structure.

[0420] Next, the action sequence can be mapped into an operation command that is recognizable by the target device and compatible with the system used by the target device. During the mapping process, each key text in the action sequence can be mapped into a command line written in a programming language that is compatible with the system language of the target device, allowing it to be called and recognized by the target device.

[0421] Then, according to the operation command, the simulated input device can input the parameters carried in the operation command to the target object and then perform the corresponding operation action, thereby completing the test of the target object.

[0422] With this embodiment, the received test information is parsed, the parsed key text is converted into operation commands for operating the target object, and the interface of the target object is called to execute the operation commands on the target object, thereby automatically completing the test operation on the target object; in this way, through parsing and mapping, the user-defined test information can be directly converted into executable operation commands, thereby, the tester does not need to write test scripts, thereby reducing the requirements for the tester; compared with writing scripts, the time length of customizing test information is greatly shortened, thereby improving the efficiency of automatic testing, and then the manpower cost and time cost required for inspection can be greatly reduced.

[0423] The process of acquiring and parsing the test information may refer to the description of the embodiment of the first device mentioned above, and will not be elaborated here.

[0424] When mapping an action sequence into an operation command adapted to a target object, this can also be done based on multiple library files, each of which contains pre-stored command lines corresponding to different keywords under the same attribute. Specifically, for each key text in the action sequence, the key text can be mapped to a library file corresponding to the attribute of the key text to obtain a command line corresponding to the key text; wherein the library file includes multiple preset command lines corresponding to multiple keywords under the same attribute, and different types of library files correspond to key texts with different attributes in the action sequence; and based on the command lines corresponding to each of the multiple key texts, an operation command is generated.

[0425] In this embodiment, the attributes of the key text can be divided into object attributes, element attributes and action attributes, wherein for each attribute, there can be multiple library files, and different library files can correspond to different operating systems. For example, for the object library file of the object attribute, it can include an object library file adapted to the Android system, and it can also include an object library file adapted to the DOS system, so that for different systems, it can obtain a mapping adapted to the system. Among them, the library file can be regarded as the representation of the various functions possessed by the input device in computer language, that is, the numerous command lines in the library file simulate the numerous functional operations of the input device. For example, for the library file of the action attribute, it can simulate the numerous operations that can be performed by the mouse and keyboard, and convert the physical operations into virtual computer language operations; for the library file of the object attribute, it can simulate the positioning operation of the mouse, and convert the mouse cursor positioning into a virtual computer language positioning; for the library file of the element attribute, it can simulate the keyboard typing operation.

[0426] During mapping, the key text can be compared with the keywords in the library file to locate the preset command line where the key text is located, and then the preset command line is used as the command line for the keyword. Among them, the multiple preset command lines included in the library file for each attribute can all be command lines for the same attribute, and the preset command lines corresponding to different keywords can be different.

[0427] The mapped command lines can be combined according to the structure of "action+object+parameter" to obtain the operation command. The more specific process of obtaining the operation command can be the same as that described in the embodiment of the first device above, and will not be repeated here.

[0428] In some exemplary embodiments, when calling the interface where the target object is located and executing an operation command on the target object, if the target object includes an object in a web page, the interface of the web page can be called to simulate the input device of the device where the target object is located to perform an operation corresponding to the operation command on the target object; if the target object includes a writing page, the interface of a second device or mechanical device can be called to perform virtual writing or physical writing on the writing page according to the operation command line.

[0429] In this embodiment, the target object can be roughly divided into an element object located in a web page, and a writing page. Among them, when the target object is an element object located in a web page and a writing page, its execution process can be similar to the execution process of the above-mentioned first device, and will not be repeated here.

[0430] In some exemplary embodiments, the target object includes an object in a web page, and the test information further includes target identification information describing the target object; before the step of calling the interface where the target object is located and executing the operation command on the target object, the method may further include the following steps:

[0431] Step S2021: Obtain identification information of each object in the currently displayed web page; wherein the identification information includes image information and / or name information of the object, and the target identification information includes appearance image and / or name information of the target object;

[0432] Step S2022: Match the target identification information with the multiple identification information respectively to obtain the position of the target object in the currently displayed web page.

[0433] This embodiment involves a process of locating the target object before executing an operation command. After locating the target object, the page position of the target object, the directory position of the device where it is located, etc. can be obtained. Therefore, the page position of the target object and the directory position of the device where it is located can be used as key values ​​as feedback for the operation command.

[0434] The device where the target object is located can be the first device or the second device described above, and the target identification information can be an appearance image of the target object. Thus, the identification information can also be the appearance images of each element. When the two are matched, the similarity between them can be determined, and the location of the target object can be determined based on the similarity. Both the target identification information and the identification information can be textual information. For example, the target identification information can be the name of the target object, and the element identification information can be the name of the element. Thus, during the comparison, the location of the target object can be determined based on the consistency between the names.

[0435] Specifically, when obtaining identification information of each object in the currently displayed web page, it can be obtained from the object library file. That is, the identification information of each object in the web page can be stored in the object library file, thereby facilitating the positioning of the target object.

[0436] Among them, the positioning of the target object can include image-based positioning and name-based positioning. In image-based positioning, the currently displayed page of the web page can be screenshoted, and the screenshot image can be matched with the appearance image of the target object to obtain the position of the target object in the currently displayed web page. Alternatively, the text in the screenshot image can be recognized, and the recognized text can be matched with the name of the target object to obtain the position of the target object in the currently displayed web page. Either or both of these methods can be executed.

[0437] Specifically, these two methods may be similar to the process of locating the target object in the first device embodiment, and will not be described in detail here.

[0438] As shown in Figure 1, its automatic test can also include a test scenario of a terminal device, in which the target object can be a writing page, and the writing page can complete writing in response to a touch on the touch display screen. In an exemplary embodiment of the method, the target object can include a writing page, and the test information can include first test information of a writing test type. When multiple key texts related to the test are parsed out in the test information, the display module of the first device can be controlled to display a preset writing setting page in response to the arrival of the first test information; a plurality of writing templates are preset in the writing setting page, and each writing template includes at least one of the following setting items: line, line width, color, font, background; and, in response to the setting information received by the writing setting page, writing data and track point data are generated; and the writing data and track point data are used as key texts.

[0439] In the writing test indicated in this example, the test information also carries a label of the writing test type. In response to the label, it can be clearly stated that it is a writing test. In order to realize the automation of the writing test, a writing setting page can be output. The writing setting page can help testers plan writing trajectories and writing styles. Specifically, the writing setting page is pre-set with multiple writing templates, and each writing template includes at least one of the following setting items: line, line width, color, font, background; the tester can select a writing template and set the writing style in the writing template.

[0440] The track point data can be randomly generated based on a selected writing template, and the writing data can be generated based on a writing style selected by the tester in the writing template. The track point data can indicate multiple writing locations on a writing page, and the writing data can generate a graphic that conforms to the writing style based on the multiple writing locations. The writing style can specify the line width, line shape, font, color, and other styles of the graphic.

[0441] Among them, the formation of the writing position point can depend on the touch position on the touch screen on which the writing page is based. For example, a physical touch can be performed on the touch screen by a robotic arm, and the touch performed by the robotic arm depends on the track point data; or it can rely on the virtual touch electrical signal received by the touch chip, that is, a virtual touch electrical signal of the corresponding touch position is generated according to the track point data, and the touch electrical signal is sent to the touch chip, so that the touch chip mistakenly believes that a real touch has occurred at a certain position point; this method is also called virtual touch, that is, virtual writing.

[0442] Among them, the specific introduction of the writing setting page can be similar to the writing setting page in the first device, and will not be repeated here.

[0443] During the automatic testing process, the test process can be monitored, and test results can be obtained, and the test results can be compared with expected test results to determine whether the expected test results are achieved. Specifically, the test information can include assertion information, which is used to indicate the expected test results of the target object. The test process of the target object can also be detected to obtain process information, and the process information can be sent to the communication terminal. The target object's interface can be called to obtain the test results of the target object after the test is completed. The test results and assertion information can be matched, and the matched results can be displayed and / or sent to the communication terminal.

[0444] In this embodiment, when detecting the test process, the operation log of the target object in the device where it is located can be detected to obtain process information. The process information can reflect the operating status of the target object when it is in the test process, which may include the above-mentioned operation log. Of course, in some other scenarios, the operating status of the target object and the hardware and software performance of the device where the target object is located can also be detected to monitor the hardware and software performance of the operating environment where the target object is located.

[0445] Among them, the method of obtaining the test results can be determined according to the type of the target object. For example, for a target object whose test results can be observed from the display interface, a screenshot of the display interface of the device where the target object is located can be obtained, and the test results can be obtained based on the screenshot; for a target object whose test results cannot be observed from the display interface, the interface where the target object is located can be called to obtain the status data of the target object after the test from the device where the target object is located, thereby obtaining the test results.

[0446] Specifically, the specific process of obtaining the test results may be similar to the process of obtaining the test results in the first device, and will not be described in detail here.

[0447] Accordingly, if the test information includes first test information of a writing type, which indicates that a writing test is being performed on a writing page in the display component, the test result can be obtained through image recognition to match the assertion information. Specifically, the interface of the display component can be called to obtain an image of the display interface of the display component after the writing test is completed, wherein the display interface image includes the writing trace. At least one of the following elements of the writing trace can be identified: font, line width, color, graphic shape, and pen shape. Then, based on the recognition result and the assertion information, the test result can be obtained.

[0448] Specifically, the process of obtaining the test results in the written test may be similar to the process of obtaining the test results in the written test in the first device, and will not be elaborated here.

[0449] In some examples, a target object may have multiple test items. As shown in FIG10 , in the login test, the test items include a test item for user name validity verification and a test item for password verification. The test item for user name validity verification may further include null value verification, length verification, letter and character verification, etc. In practice, it is necessary to design test information for each test item related to verification in the login item. In some embodiments of this example, based on a basic test information, the test information required for multiple test items included in the test item to which the test information belongs can be automatically generated to improve the design efficiency of test information for multiple test items included in the same test item. Accordingly, after the target object is automatically tested based on the test information, multiple target preset test parameters matching the target test type carried in the test information can be obtained from the use case database; and based on each target preset test parameter, multiple basic test information associated with the test information is generated; and in response to calling any one of the multiple basic test information, the object to be tested is automatically tested according to the called basic test information.

[0450] Among them, the use case database includes multiple test types and multiple preset test parameters corresponding to each test type, wherein different preset test parameters correspond to different test items under the same test type; wherein different basic test information is used to implement different test items under the test information.

[0451] Specifically, the process of automatically generating basic test information based on the test information may be similar to the process of generating basic test information in the first device, and will not be described in detail here.

[0452] An embodiment of the present disclosure further provides an electronic device, wherein a computer program stored in the electronic device enables a processor to execute the testing method as described in the embodiment of the present disclosure.

[0453] The present disclosure also provides a non-transitory computer-readable recording medium having a program recorded thereon. When the program is executed by a processor in a first device, the program automatically tests a target object on the first device or a second device. The automatic testing includes:

[0454] Acquire test information sent to the first device; wherein the test information is used to describe the test items to be performed on the target object;

[0455] Parsing multiple key texts related to the test from the test information, and constructing the multiple key texts into an action sequence of a target data structure, wherein the action sequence is used to represent an operation flow performed on the target object when executing the test content;

[0456] The action sequence is mapped into an operation command adapted to the target object, and the operation command is executed on the target object to complete the test indicated by the test information on the target object.

[0457] The non-transitory computer-readable recording medium may be located on the first device, and may automatically test the target object located on the first device or the second device in response to the test information uploaded by the user.

[0458] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0459] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity, or device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, commodity, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, commodity, or device that includes the element.

[0460] The first device, the second device, the testing method, the system and the medium provided by the present disclosure are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core idea of ​​the present disclosure. At the same time, for those skilled in the art, according to the idea of ​​the present disclosure, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

[0461] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0462] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0463] References herein to "one embodiment," "an embodiment," or "one or more embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Furthermore, please note that instances of the phrase "in one embodiment" do not necessarily all refer to the same embodiment.

[0464] In the description provided herein, numerous specific details are described. However, it is understood that embodiments of the present disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques are not shown in detail so as not to obscure the understanding of this description.

[0465] In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present disclosure may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0466] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present disclosure.

Claims

1. A first device, wherein, the first device includes a processor, and a program runs on the processor. When the program runs, it automatically tests a target object on the first device or a second device. The automatic test includes: obtaining test information sent to the first device, where the test information is used to describe the test content to be performed on the target object; parsing multiple key texts related to the test from the test information, and constructing the multiple key texts into an action sequence of a target data structure, where the action sequence is used to represent the operation process of the target object when performing the test content; mapping the action sequence to an operation command adapted to the target object, and executing the operation command on the target object to complete the test indicated by the test information for the target object.

2. The first device according to claim 1, wherein, the first device further includes a memory, and multiple library files associated with the program are stored in the memory. Different library files correspond to key texts with different attributes in the action sequence. The step of mapping the action sequence to an operation command adapted to the target object includes: for each key text in the action sequence, mapping the key text to a library file corresponding to the attribute of the key text to obtain a command line corresponding to the key text; wherein, the library file includes multiple preset command lines corresponding to multiple keywords; generating the operation command based on the command lines corresponding to the multiple key texts respectively.

3. The second device according to claim 2, wherein, the attribute includes an action attribute, an element attribute, and an object attribute. The key text of the object attribute is used to identify the target object; wherein, the multiple library files include an action library file corresponding to the action attribute, a test database file corresponding to the element attribute, and an object library file corresponding to the object attribute; wherein, the action library file includes preset command lines corresponding to multiple operation actions, the test database file includes preset command lines corresponding to different input parameters, and the object library file includes preset command lines for positioning different objects.

4. The first device according to claim 1, wherein, the step of parsing multiple key texts related to the test from the test information includes: segmenting the test information to obtain multiple paragraphs; extracting the key text from each paragraph based on a preset blacklist and / or whitelist.

5. The first device according to claim 1, wherein, the step of executing the operation command on the target object includes: when the target object includes an object in a web page, calling the interface of the web page to simulate the input device of the device where the target object is located to perform an operation corresponding to the operation command on the target object; when the target object includes a writing page, calling the interface of the writing page on the writing page Perform virtual writing according to the operation command line, or call the interface of the mechanical device to perform physical writing on the writing page according to the operation command line.

6. The first device according to claim 1, wherein, the target object includes an object in a web page, and the test information further includes target identification information describing the target object. Before the step of performing the operation command on the target object, when the program runs in the processor, it is also used to locate the target object; The locating of the target object includes: Obtain the identification information of each object in the currently displayed web page; Match the target identification information with the multiple identification information respectively to obtain the position of the target object in the currently displayed web page.

7. The first device according to claim 6, wherein, the target identification information includes the appearance image and / or name of the target object. The step of calling the device where the target object is located to obtain the identification information of each object in the currently displayed web page includes: Take a screenshot of the currently displayed web page, and perform image segmentation on the captured image to obtain multiple image regions; wherein, different image regions correspond to different objects; Perform at least one of the following processes on the image regions to obtain the identification information corresponding to each object: Perform character recognition on the image region to obtain the identification information of the text type of the object for matching with the name; Extract features from the image region to obtain the identification information of the feature representation type for matching with the feature map of the appearance image.

8. The first device according to claim 2, wherein, the first device further includes a display module. The target object includes the writing page, and the test information includes first test information for performing a writing test. The step of parsing multiple key texts related to the test from the test information includes: In response to the first test information, control the display module to display a preset writing setting page; a variety of writing templates are preset in the writing setting page, and each writing template includes at least one of the following setting items: line, line width, color, font, background; Generate writing data in response to the selected writing template; and generate trajectory point data in response to the setting information received in the selected writing template; Determine the writing data and the trajectory point data as the key texts; The step of performing the operation command on the target object includes: Send the writing data to the writing page, and call the display component of the device where the writing page is located, so that the writing page responds to the received physical writing signal or virtual writing signal and generates a writing trajectory on the writing page. Wherein, the physical writing signal is a touch signal triggered by a mechanical device connected to the first device on the writing page based on the trajectory point data; the virtual writing signal is a virtual touch signal sent by the display component to the writing page based on the trajectory point data.

9. The first device according to claim 1, wherein, the test information includes assertion information, and the assertion information is used to indicate the expected test result of the target object; when the program is run by the processor, it is further used to perform the following steps: Detect the test process of the target object to obtain process information, and send the process information to a communication terminal communicating with the first device; Obtain the test result of the target object after the test is completed; Match the test result and the assertion information, and output the matching result to the display module of the first device for display, and / or send the matching result to the communication terminal.

10. The first device according to claim 9, wherein, the target object includes an object in a web page; detecting the test process of the target object includes detecting at least one of the following: Script data of the current web page, the display interface of the current web page, the status information of the current web page, the frame rate of the current web page, the operation log of the current web page.

11. The first device according to claim 9, wherein, Detecting the test process of the target object further includes detecting at least one of the following performances of the first device: the running efficiency of the processor of the device where the target object is located, the memory usage rate, the data traffic of the current web page, the power consumption of the device where the target object is located, the usage rate of the graphics processor of the device where the target object is located, the network connection status of the device where the target object is located.

12. The first device according to claim 9, wherein, the target object includes a display component, the test information includes first test information of a writing type, and the first test information is used to indicate a writing test on a writing page; The step of obtaining the test result of the target object after the test is completed includes: Obtain an image of the display interface after the writing test is completed, and the display interface image includes a writing trajectory; Identify at least one of the following contents of the writing trajectory: font, line width, color, graphic shape, and pen shape; Based on the recognition result and the assertion information, obtain the test result.

13. The first device according to claim 1, wherein, The memory on the first device also stores a use case database, and the use case database includes multiple test types and multiple preset test parameters corresponding to each test type; wherein, different preset test parameters correspond to different test items under the same test type; when the program is executed by the processor, it is further used to implement the following steps: Based on the target test type carried in the test information, obtain multiple target preset test parameters that match the target test type from the use case database; Based on each of the target preset test parameters, generate multiple basic test information associated with the test information; wherein, different basic test information is used to implement different test items under the test information; Send the multiple basic test information to the use case source connected to the first device.

14. The first device according to claim 1, wherein, the test information includes a target field, and the target field is used to indicate the second test information on which the current test information depends; when the program is executed by the processor, the following steps are further implemented: In response to the content of the target field, obtain the second test information; Perform the test indicated by the second test information on the target object; After the second test information test is completed, execute the step of performing the operation command on the target object to complete the test indicated by the test information on the target object.

15. A test method, wherein, the method includes: Obtain the test information to be executed; wherein, the test information is used to describe the test items to be performed on the target object; Parse multiple key texts related to the test from the test information, and construct the multiple key texts into an action sequence of a target data structure, where the action sequence is used to characterize the operation process of the target object when the test content is executed; Map the action sequence to an operation command adapted to the target object, and execute the operation command on the target object to complete the test indicated by the test information on the target object.

16. The test method according to claim 15, wherein, the mapping of the action sequence to an operation command adapted to the target object includes: For each of the key texts in the action sequence, map the key text to a library file corresponding to the attribute of the key text to obtain a command line corresponding to the key text; wherein, the library file includes multiple preset command lines corresponding to multiple keywords under the same attribute, and different library files correspond to key texts with different attributes in the action sequence; Generate the operation command based on the command lines corresponding to the multiple key texts respectively.

17. The test method according to claim 16, wherein, the step of calling the interface where the target object is located and executing the operation command on the target object includes: When the target object includes an object in a web page, call the interface of the web page to simulate the input device of the device where the target object is located to perform an operation corresponding to the operation command on the target object; When the target object includes a writing page, call the interface of the writing page to execute a virtual book on the writing page according to the operation command line, or call the interface of a mechanical device to perform an entity writing on the writing page according to the operation command line.

18. The test method according to claim 16, wherein, The target object includes an object in a web page, and the test information further includes target identification information describing the target object; Before the step of calling the interface where the target object is located and executing the operation command on the target object, the method further includes: Calling the device where the target object is located to obtain the identification information of each object in the currently displayed web page; wherein, the identification information includes the image information and / or name information of the object, and the target identification information includes the appearance image and / or name information of the target object; Matching the target identification information with the multiple pieces of identification information respectively to obtain the position of the target object in the currently displayed web page.

19. The test method according to claim 16, wherein, The target object includes a display component, the test information includes first test information of a writing test type, and parsing multiple key texts related to the test from the test information includes: In response to the first test information, a preset writing setting page is displayed; a variety of writing templates are preset in the writing setting page, and each writing template includes at least one of the following setting items: line, line width, color, font, background; Generating writing data in response to the selected writing template; and generating trajectory point data in response to the setting information received in the selected writing template; Generating a writing trajectory on the writing page based on the trajectory point data and the writing data.

20. The test method according to claim 15, wherein, The test information includes assertion information, and the assertion information is used to indicate the expected test result of the target object; the method further includes: Detecting the test process of the target object to obtain process information and sending the process information to a communication terminal; Calling the interface of the target object to obtain the test result of the target object after the test is completed; Matching the test result with the assertion information, and displaying the matching result, and / or sending the matching result to the communication terminal.

21. The test method according to claim 20, wherein, The test information includes first test information of a writing type, and the first test information is used to indicate a writing test on a writing page; obtaining the test result of the target object after the test is completed includes: Obtaining a display interface image of the writing page after the writing test is completed, and the display interface image includes a writing trajectory; Identifying at least one of the following contents of the writing trajectory: font, line width, color, graphic shape, and pen shape; Obtaining the test result based on the identification result and the assertion information.

22. The test method according to claim 15, wherein, The method further includes: Based on the target test type carried in the test information, obtain multiple target preset test parameters that match the target test type from the use case database; wherein, the use case database includes multiple test types and multiple preset test parameters corresponding to each test type, and different preset test parameters correspond to different test items under the same test type; Based on each of the target preset test parameters, generate multiple basic test information associated with the test information; wherein, different basic test information is used to implement different test items under the test information; In response to the invocation of any one of the multiple basic test information, perform an automatic test on the object to be tested as indicated by the invoked basic test information.

23. A second device, wherein, The second device is configured with a target object, a display component, and a program that supports the operation of the target object. When the program is run by the processor of the second device, it is used to perform an automatic test indicated by a first device connected to the second device. The automatic test includes: Receive an operation command sent by the first device through the interface; Simulate an input device to perform an operation corresponding to the operation command on the target object; Display the screen after executing the operation command on the display component; wherein, the operation command is obtained by mapping an action sequence after standardizing the test information when the processor in the first device runs the program, and the action sequence is used to characterize the operation process of the target object when executing the test content.

24. The second device according to claim 23, wherein, The operation command includes trajectory point data and writing data. The display component is configured with a touch screen, and the target object includes a writing page. Performing an operation corresponding to the operation command on the target object includes: Based on the trajectory point data, simulate a virtual touch of a user's finger on the touch screen, and based on the writing data, generate a writing trajectory corresponding to the position of the virtual touch on the writing page; Or, based on the operation command, in response to the touch of a mechanical device connected to the first device on the touch screen, generate a writing trajectory corresponding to the touch position on the writing page based on the writing data; wherein, the mechanical device is used to perform a touch on the touch screen based on the trajectory point data sent by the first device.

25. The second device according to claim 23, wherein, The second device is configured with an operating device. Performing an operation corresponding to the operation command on the target object includes at least one of the following: Simulate continuous touches of the operating device on the page buttons of the web page to cause a change in at least one of the following of the display page: display brightness, page opening state, and page size; Simulate continuous touches of the operating device on the switch of the communication button on the second device to cause a change in the network connection state of the second device; Simulate the operation device to switch the signal source connected to the second device to jump from one signal source to another signal source; The signal source is used to provide a display page.

26. A test system includes a first device and a second device. The first device is configured with a first page, and the second device is configured with a target object and a second page; Wherein, The first device is configured to, in response to an automatic test triggered on the first page, obtain test information, convert the test information into an action sequence, and map the action sequence to an operation command adapted to the target object; wherein, the test information is used to describe the test items to be performed by the target object, and the action sequence is used to characterize the operation process of the target object when performing the test content; The second device is configured to, in response to the first device's call to the interface where the target object is located, execute the operation command on the target object and display the execution process and execution result of the executed operation command on the second page.

27. A non-transitory computer-readable recording medium, Wherein, The computer-readable recording medium records a program. When the program is executed by a processor in the first device, it automatically tests the target object on the first device or the second device. The automatic test includes: Obtain the test information sent to the first device; wherein, the test information is used to describe the test items to be performed by the target object; Parse multiple key texts related to the test from the test information, and construct the multiple key texts into an action sequence of a target data structure. The action sequence is used to characterize the operation process of the target object when performing the test content; Map the action sequence to an operation command adapted to the target object, and execute the operation command on the target object to complete the test indicated by the test information on the target object.

Images