Test script acquisition method and device and electronic equipment

Abstract

The invention discloses a test script obtaining method and device and electronic equipment, and belongs to the technical field of automatic testing. The method comprises the following steps: analyzing a test case execution step video to obtain an action data set of each operation in the test case execution step video; identifying the action data set of each operation to obtain the action category of each operation; obtaining step data of each operation according to the action data set of each operation and the action category of each operation; according to the step data of each operation, generating a test code corresponding to each operation; and combining the test codes corresponding to the operations in sequence to obtain a test script of the test case execution step video. In this way, the efficiency can be improved, and the cost can be reduced.

Description

Technical Field

[0001] The present application belongs to the field of automated testing technology, and specifically relates to a test script acquisition method, device and electronic equipment. Background Art

[0002] Among the related technologies, the mainstream technical solutions for user interface (UI) automated testing are Selenium and Appium, which mainly implement automated testing by having test developers write python or java scripts according to the steps of automated test cases. This is not only inefficient but also costly. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a test script acquisition method, device and electronic device, which can solve at least one of the problems of low efficiency and high cost when acquiring test scripts.

[0004] In the first aspect, an embodiment of the present application provides a test script acquisition method, the method comprising: obtaining an action data set of each operation in the test case execution step video by analyzing the test case execution step video, wherein the action data set includes at least one operation data corresponding to the operation; obtaining an action category of each operation by identifying the action data set of each operation, wherein the action category is used to characterize the test step corresponding to the operation; obtaining step data of each operation according to the action data set of each operation and the action category of each operation, wherein the step data is used to characterize the overall action corresponding to the operation; generating a test code corresponding to each operation according to the step data of each operation; and obtaining a test script of the test case execution step video by combining the test codes corresponding to each operation in sequence.

[0005] In the second aspect, an embodiment of the present application provides a test script acquisition device, which includes: a first acquisition module, used to acquire an action data set of each operation in the test case execution step video by analyzing the test case execution step video, wherein the action data set includes at least one operation data corresponding to the operation; a second acquisition module, used to acquire an action category of each operation by identifying the action data set of each operation, wherein the action category is used to characterize the test step corresponding to the operation; a third acquisition module, used to acquire step data of each operation based on the action data set of each operation and the action category of each operation, wherein the step data is used to characterize the overall action corresponding to the operation; a generation module, used to generate a test code corresponding to each operation based on the step data of each operation; a fourth acquisition module, used to acquire the test script of the test case execution step video by combining the test codes corresponding to each operation in sequence.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0008] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0009] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the method described in the first aspect.

[0010] In an embodiment of the present application, the action data set of each operation in the test case execution step video can be obtained by analyzing the test case execution step video, and then the action data set of each operation can be identified to obtain the action category of each operation, and then the action data set of each operation, the action category of each operation and the interval time between adjacent operations can be combined to obtain the step data of each operation, and finally the test script of the test case execution step video can be obtained based on the step data of each operation. By analyzing the test case execution step video, the step data of each operation can be obtained, and then the test script can be obtained. Not only can the efficiency of obtaining the test script be improved, but also the cost can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A flow chart showing a method for obtaining a test script provided in an embodiment of the present application is shown; Figure 2a A schematic diagram showing the operation in the video provided by an embodiment of the present application; Figure 2b A schematic diagram showing a keyboard in a video provided by an embodiment of the present application is shown; Figure 3 A schematic diagram showing a keyboard provided in an embodiment of the present application in use and not in use; Figure 4 A schematic diagram of a Transformer model provided in an embodiment of the present application is shown; Figure 5 A schematic diagram showing the structure of a cloud test script acquisition device provided in an embodiment of the present application is shown; Figure 6 A schematic diagram showing the structure of another cloud test script acquisition device provided in an embodiment of the present application; Figure 7 A structural block diagram of an electronic device provided by an exemplary embodiment of the present application is shown.

[0012] Figure 8 A structural block diagram of an electronic device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0014] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0015] Selenium is an automated testing tool for World Wide Web (Web) applications. Appium is an open source test automation framework that can be used for native, hybrid, and mobile web application testing. Both Selenium and Appium use page attributes in the application to locate application elements and operate on the elements to achieve the purpose of automated testing. Selenium and Appium provide a recording method to generate UI automated test scripts: In Appium, you can use Appium's UI recorder function to record and generate automated test scripts. First, you need to open Appium's UI recorder, then connect the test terminal, and perform test operations on the test terminal, such as clicking, inputting, etc. After completing the test operation, the recorder will automatically generate the corresponding automated test script. For Selenium, you can use Selenium IDE to record and generate automated test scripts. Selenium IDE is a plug-in based on the Firefox browser, which can record and replay test operations and generate corresponding Selenium test scripts.

[0016] Both Appium and Selenium support the generation of test scripts through recording and playback, but the recording generation method requires connecting to the test terminal, and then manually executing the test cases step by step according to the test cases, and generating test scripts step by step. After each step of the script generation, it is necessary to manually confirm the correctness and debug it. Testers need to spend a lot of energy to optimize the case execution and script recording. Python is a widely used interpreted, high-level and general programming language, and Java is an object-oriented programming language.

[0017] The test script acquisition method provided in the embodiment of the present application is described in detail below through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0018] Figure 1A test script acquisition method provided by an exemplary embodiment of the present application is shown. The method 100 can be executed by an electronic device. In other words, the method can be executed by software or hardware installed in the electronic device. The electronic device can be a terminal device such as a mobile phone. Figure 1 As shown, the method mainly includes the following steps: S101: Analyze the test case execution step video to obtain an action data set of each operation in the test case execution step video.

[0019] The action data set includes at least one piece of operation data corresponding to the operation.

[0020] In an embodiment of the present application, the action data set of each operation in the test case execution step video can be obtained by analyzing the test case execution step video, and the action data set includes at least one operation data corresponding to the operation. By analyzing the test case execution step video, at least one basic operation data of each operation can be obtained, for example, the start video frame and the end video frame of the operation, whether the operation involves the keyboard, etc., and then the action data set of each operation can be obtained, and the operation data that can be extracted from each operation can be recorded and counted to avoid losing key information and facilitate further analysis of each operation.

[0021] In an optional implementation, the step of analyzing the test case execution step video to obtain the action data set of each operation in the test case execution step video may include the following steps: Step 1, obtain the test case execution step video in a predetermined format, wherein the test case execution step video includes operation labels and keyboard labels. In the embodiment of the present application, the test case execution video needs to be recorded in a prescribed manner: start recording the video after turning on the terminal developer option - display tap operation feedback, and after recording the video, configure the icon displayed by tapping in the video as a step operation label. Figure 2a as well as Figure 2b As shown, after turning on the display of tap operation feedback, the system's click mark is Figure 2a In the box part, the graphic of this part is configured as the operation label operateSign; at the same time, the keyboard label keyboradSign of the terminal is configured. The video obtained in this way can better identify each operation, which helps to improve efficiency. In an embodiment of the present application, by setting the keyboard label, the operation behaviors in different contexts can be intelligently distinguished, such as distinguishing between "click" and "input" actions. This method of comprehensively considering the keyboard status can improve the accuracy and context sensitivity of action recognition, especially the performance when dealing with complex interactive scenarios.

[0022] Step 2: Analyze the test case execution step video frame by frame using image recognition technology to obtain the action data set of each operation. In an embodiment of the present application, image recognition technology can be used to analyze the test case execution step video frame by frame, identify and record each frame according to the frame number, and obtain the action data set of each operation, which is not only more detailed but also can avoid omissions.

[0023] In the embodiment of the present application, since the test case execution step video can be obtained by recording the screen through the recording tool provided by the terminal device, compared with the related art that requires additional development of recording tools and installation on the test device used by the tester to record the operation video, the embodiment of the present application does not require additional development of recording tools. Moreover, the test case execution step video can be analyzed by using image recognition technology, without the need for manual recognition, so the difficulty of generating test scripts can be effectively reduced, further improving the test efficiency.

[0024] In an optional implementation, the operation includes a non-input operation and an input operation; The step of performing frame-by-frame analysis on the test case execution step video by using image recognition technology to obtain the action data set of each operation includes: When it is recognized that there is an operation label in the first frame and there is no operation label in the second frame, it is determined that the first frame to the second frame is a non-input operation and an action data set of the non-input operation is obtained, wherein the order of the second frame is greater than that of the first frame, and the action data set of the non-input operation includes at least one first operation data, and the format of the first operation data is {A, frame, keyboard_use}, A is the center position of the operation label in the third frame, frame is the frame number of the third frame, when there is a keyboard label in the third frame, keyboard_use is 1, otherwise, keyboard_use is 0, and the third frame is a video frame whose order is greater than or equal to the first frame and less than the second frame.

[0025] In the embodiment of the present application, if it is detected that the operation label operateSign appears in the first frame (current frame) N, then the operation record action1 begins, and the current frame number frame and the center position (x, y) of the operation label operateSign are recorded. At the same time, the frame where the operation label appears is identified by the keyboardSign label. If the keyboard appears in the frame, keyboard_use=1 is recorded. If the keyboard does not appear, keyboard_use=0 is recorded, and {(x, y), frame, keyboard_use} is recorded in the first operation data of the acttion1 array. Next, it is necessary to check the N+1 frame to check whether the operation label operateSign appears in the N+1 frame. If the operation label appears in the N+1 frame, the N+1 frame also belongs to action1. After image recognition processing, the corresponding {(x, y), frame, keyboard_use} of the N+1 frame is recorded in the second operation data of the acttion1 array. Check the next frame in sequence until the operation label operateSign does not appear in the second frame (N+k frame). Then the Nth frame to the N+k-1th frame is a continuous operation action, and the recorded acttion1 array is the action data set of this operation.

[0026] In the above optional implementation, the step of analyzing the test case execution step video frame by frame by using image recognition technology to obtain the action data set of each operation also includes: When it is identified that the fourth frame has an operation label and a keyboard label, and there is no operation label and keyboard label in the fifth frame, it is determined that the fourth frame to the fifth frame are an input operation and an action data set of the input operation is obtained, wherein the order of the fifth frame is greater than the fourth frame, the action data set of the input operation includes at least one second operation data, and the format of the second operation data is {A, frame, keyboard_use}, A is the center position of the operation label in the sixth frame, frame is the frame number of the sixth frame, when there is a keyboard label in the sixth frame, keyboard_use is 1, otherwise, keyboard_use is 0, and the sixth frame is a video frame whose order is greater than or equal to the fourth frame and less than the fifth frame.

[0027] In the embodiment of the present application, if it is detected that the keyboard label keyboradSign appears in the fourth frame (current frame) N, then the input operation record action2 begins, and the current frame is checked to see if the record operation label operateSign appears. If so, the current frame number frame and the center position (x, y) of the operation label operateSign are recorded, and {(x, y), frame, keyboard_use} are recorded in the first operation data of the acttion2 array. If the keyboard label keyboradSign does not appear, it is not recorded. Next, the N+1 frame needs to be checked to see if the keyboard label keyboradSign appears in the N+1 frame. If the keyboard label appears in the N+1 frame, the N+1 frame also belongs to the input operation action2. After image recognition processing, if the operation label operateSign appears in the N+1 frame, the corresponding {(x, y), frame, keyboard_use} of the N+1 frame is recorded in the second operation data of the acttion2 array. If the operation label operateSign does not appear in the N+1 frame, it is not recorded. The next frame is checked in sequence until the operation label operateSign does not appear in the N+kth frame, and the Nth frame to the N+k-1th frame are a continuous operation action.

[0028] S102: Acquire the action category of each operation by identifying the action data set of each operation, wherein the action category is used to characterize the test step corresponding to the operation.

[0029] In the embodiment of the present application, the action category of each operation can be obtained by identifying the action data set of each operation, wherein the action category is used to characterize the test step corresponding to the operation (slide left, slide right, slide up, slide down, click, long press and input). It can more accurately obtain which test step the operation corresponding to the action data set belongs to, which is convenient for generating test scripts.

[0030] In an optional implementation, the step of identifying the action data set of each operation to obtain the action category of each operation may include the following steps: Step 1, extracting features from the action data sets of each operation to obtain the target features of each operation, wherein the target features include: change amount, action duration, keyboard usage status, starting position and ending position; in the embodiment of the present application, feature extraction can be performed on the action data sets of each operation, and target features can be extracted from the actions of each operation, and these target features include: change amount deltas, action duration duration, keyboard usage status keyboard_use, starting position start_position and ending position end_position. By extracting features from the action data sets of each operation, the action data sets can be streamlined, and key target features can be extracted from them to obtain the action categories of each operation, which can improve efficiency and save recognition accuracy.

[0031] Step 2, by using the trained action category recognition model to perform action recognition on the target features of each operation, to obtain the action category of each operation. In an embodiment of the present application, by using the trained action category recognition model to perform action recognition on the target features of each operation, to obtain the action category of each operation, it is not only efficient but also helps to reduce costs.

[0032] In an optional implementation, for step 1, extracting features from action data sets of each operation to obtain target features of each operation may include (regardless of the order of execution): (1) Obtain the change amount of each operation by calculating the coordinate difference between each frame and the previous frame in the action data set of each operation; in the embodiment of the present application, the change amount deltas is the coordinate change amount between consecutive frames, which can be achieved by calculating the difference between the x and y coordinates of each frame and the previous frame. Deltas can help the model understand the direction of cursor movement. For example, the positioning y coordinate value of each frame in the operation coordinates of the right slide operation is unchanged, while the x coordinate value gradually increases, that is, the change amount deltasY of the y coordinate of the right slide operation is always 0, and the change amount deltasX of the x coordinate is always a positive number; while for the upward slide operation, the change amount deltasX of the x coordinate is always 0, and the change amount deltasY of the y coordinate is always a positive number; in particular, if an operation is not simply an up, down, left, or right slide operation, then deltasX and deltasY will not have a value that is always 0.

[0033] (2) The action duration of each operation is obtained by calculating the difference between the frame value of the last frame and the frame value of the first frame in the action data set of each operation. In the embodiment of the present application, the action duration can be calculated by subtracting the frame value of the first frame from the frame value of the last frame to distinguish between fast actions and slow actions. At the same time, click and long press operations with unchanged position can also be distinguished by the duration of the operation.

[0034] (3) Obtain the keyboard usage status of each operation through the action data set of each operation; in the embodiment of the present application, the value of the keyboard_use field in the action data set can be directly used as a feature, that is, the keyboard usage status, to distinguish whether the keyboard is used during the operation. Whether the keyboard is used is used to distinguish whether it is a text input operation, such as Figure 3 As shown, the touch operation when the keyboard pops up is usually closely related to text input, while the touch operation when the keyboard does not pop up is more related to screen navigation or other control actions.

[0035] (4) Obtain the starting position and ending position of each operation through the coordinates of the first frame and the last frame in the action data set of each operation. In the embodiment of the present application, the starting position (coordinates of the first frame) and the ending position (coordinates of the last frame) of the operation are also important features that can be used to identify the overall trend of the operation.

[0036] In an optional implementation, for step 2, the action category recognition model can be trained in the following manner: The first step is to obtain training data. In an embodiment of the present application, training data may be obtained first, and the training data includes target features of samples and action category labels.

[0037] The second step is to obtain target training data by preprocessing the training data. In the embodiment of the present application, the training data can be standardized so that its mean is 0 and its variance is 1 to improve the stability and efficiency of model training. Then, the corresponding action category labels (such as "swipe_up", "swipe_down", etc.) are converted into integers or one-hot encodings for model processing.

[0038] The third step is to obtain a feature vector corresponding to the target training data by inputting the target training data into the encoder; in the embodiment of the present application, the action category recognition model can be a Transformer model, which is an encoder-decoder model of an attention mechanism, and the structure diagram of the model is as follows Figure 4As shown, an encoder ActionRecognitionEncoder can be designed first, and the encoder can convert the target training data into a feature vector. In this way, the feature vector corresponding to the target training data can be obtained by inputting the target training data into the encoder.

[0039] The fourth step is to obtain the action category corresponding to the feature vector by inputting the feature vector into the decoder; in the embodiment of the present application, a decoder ActionRecognitionDecoder can be designed for the model to convert the feature vector input by the encoder into an action category. The decoder uses a series of fully connected layers, combined with the Rectified Linear Unit (ReLU) activation function to process and refine the feature representation provided by the encoder, and extract high-level features. At the same time, in order to reduce the risk of overfitting, a Dropout layer (a structure that can be used to reduce overfitting of neural networks) is embedded behind each fully connected layer. Finally, a fully connected layer is used as a classifier to map high-level features to the probability distribution of action categories, completing the prediction and classification from features to action categories.

[0040] The fifth step is to obtain the trained action category recognition model by training the encoder and the decoder. The model is trained using the above-mentioned Transformer-based encoder-decoder structure as a framework. In the embodiment of the present application, the Transformer model is used as the action category recognition model, which can make full use of computer resources and improve computing efficiency.

[0041] In an optional implementation, for the second step, obtaining the feature vector corresponding to the target training data by inputting the target training data into an encoder may include the following steps: Step 1, obtain fused features by fusing the change amount, starting position, and ending position in the target training data; in an embodiment of the present application, other features (change amount, starting position, and ending position) except the keyboard usage status keyboard_use and the action duration duration can be fused through a fully connected layer to extract high-level representations of these features.

[0042] Step 2, by using the attention mechanism to calculate the weights and weight the keyboard usage status and action duration in the target training data, obtain the weighted keyboard usage status and action duration; in the embodiment of the present application, an attention mechanism can be set, and the two features of keyboard use keyboard_use and action duration duration are used to calculate the attention weights of the keyboard_use and action duration duration features using the attention mechanism AttentionMechanism class, so that the model can dynamically adjust the weights of these two features: the call method of the AttentionMechanism class can be designed to receive the two features of keyboard_use and duration as input, first concatenate them along the last dimension, and then perform feature transformation through a fully connected layer to extract the high-level representation of the two features after merging. Then, another fully connected layer is used to calculate the final attention weight using the sigmoid activation function. This weight value is between 0 and 1, indicating the importance of the keyboard_use feature relative to the duration feature. In the design of the model, other features except keyboard_use and duration are fused through a fully connected layer to extract the high-level representation of these features. The keyboard_use and duration are weighted and fused through the attention mechanism, so that the model can dynamically adjust the importance of the two features according to the current input context, rather than statically giving the same weight to all features. The sample code is shown in Table 1.

[0043] Table 1

[0044] In an embodiment of the present application, the weights calculated by the attention mechanism can be used to weight keyboard_use and action duration to obtain the weighted keyboard usage status and action duration.

[0045] Step 3, obtaining a merged feature by merging the weighted keyboard usage status and action duration and the fusion feature; in an embodiment of the present application, the weighted keyboard usage status and action duration and the fusion feature can be merged to obtain a merged feature.

[0046] Step 4: Process the merged features through a fully connected layer to obtain a feature vector corresponding to the target training data. In an embodiment of the present application, the merged features can be passed through an additional fully connected layer to obtain a feature vector corresponding to the target training data, which is used as the final output of the encoder, and this output will be used as the input of the decoder.

[0047] In an embodiment of the present application, multi-dimensional information of the target features of the operation (change amount, action duration, keyboard usage status, starting position and ending position) can be mined, so that the action category recognition model can more accurately capture the core attributes of the operation, more deeply understand the context of the operation, and improve the ability to distinguish different types of actions (especially between "click" and "input").

[0048] In actual applications, if when obtaining an action data set, the operations are divided into non-input operations and input operations, and their data sets are distinguished (action1 and action2), for the action data set of the input operation, the action category of the operation can be directly determined as input without being identified by the action category model, or the action data sets of all operations can be input into the action category model. The embodiments of the present application do not make specific limitations.

[0049] In practical applications, the action data sets of each operation can be identified manually, or they can be identified in the manner described in the embodiments of the present application, without specific limitation. In the embodiments of the present application, the data of the operation action can be recorded in json format, and this json contains the action category label label and the action data set action of each operation. The label can be identified manually, or it can be obtained using an action category recognition model and added to the json data. In the embodiments of the present application, the label is: swipe_up, swipe_down, swipe_left, swipe_right, tap, long press long_press, input.

[0050] Exemplarily, the json format is roughly as shown in Table 2.

[0051] Table 2

[0052]

[0053] S103: Acquire step data of each operation according to the action data set of each operation and the action category of each operation.

[0054] The step data is used to represent the overall action corresponding to the operation.

[0055] In an embodiment of the present application, the step data corresponding to each operation for characterizing the overall action of the operation can be obtained according to the action data set of each operation and the action category of each operation. The step data is a summary of the overall behavior of the operation. For example, if the category of the operation is identified as input, its input content can be obtained according to the number of frames in the action data set (the click position of the start frame is obtained as the text box position, the coordinate information (x, y), and the text recognition of the frame image is performed on the start click frame and the end click frame. At this time, the end click frame will have more string text than the start click frame, and the inconsistent text text will be recognized as the input content of this input operation), and then the input content, the coordinates in the action data set, and the action category input are combined to obtain the step data of the operation. In actual applications, the format of the step data of each operation is as follows: swipe up (swipe_up, coordinate 1, coordinate 2), swipe down (swipe_down, coordinate 1, coordinate 2), swipe left (swipe_left, coordinate 1, coordinate 2), swipe right (swipe_right, coordinate 1, coordinate 2), click (tap, coordinate), long press (long_press, click duration), input (input, coordinate, input content).

[0056] In practical applications, the step data of each operation can also be combined with the preset interval sleeptime of each operation to obtain a complete operation step string of the test case execution step video. Table 3 is an example of a complete operation step string obtained.

[0057] Table 3

[0058] S104: Generate test codes corresponding to each of the operations according to the step data of each of the operations.

[0059] In the embodiment of the present application, the test code corresponding to each operation can be generated according to the step data of each operation. This one-to-one correspondence method is more accurate and can avoid omissions.

[0060] In an optional implementation, generating test codes corresponding to each of the operations according to the step data of each of the operations includes: According to the action category in the step data of each operation, a test code corresponding to each operation is generated, wherein the action category includes: left swipe, right swipe, up swipe, down swipe, click, long press and input.

[0061] In an embodiment of the present application, the test code corresponding to each operation can be generated according to the action category in the step data of each operation. In practical applications, the corresponding operation function can be designed according to the action category, including: coordinate click function click_position(), text box output input_text_at_position(), left slide swipe_left_by_position(), right slide swipe_right_by_position(), up slide swipe_up_by_position(), down slide swipe_down_by_position() and long press long_press(). Then, the action category in the step data of each operation is connected with the corresponding operation function to generate the corresponding test code. This is not only more accurate, but also can improve efficiency.

[0062] In an optional implementation, the test code corresponding to each operation is generated according to the action category in the step data of each operation, including the following four situations: Scenario 1: When the action category of the operation is click, the test code corresponding to the operation is generated according to the coordinate data in the step data of each operation; in this embodiment of the application, when the action category of the operation is click, the coordinates (x, y) can be obtained from the step data of the operation as the input data of the click function click_position, so as to realize the click of the coordinates (x, y) on the mobile phone software (Application, APP) page and generate the corresponding test code.

[0063] Scenario 2: When the action category of the operation is one of swiping left, swiping right, swiping up, and swiping down, the test code corresponding to the operation is generated according to the coordinate data in the step data of the operation and the specific sliding direction; in the embodiment of the present application, when the action category of the operation is one of swiping left, swiping right, swiping up, and swiping down, coordinate 1 (x1, y1) and coordinate 2 (x2 and y2) can be obtained from the step data of the operation as input data for the sliding operation; then, the corresponding operation functions swipe_left_by_position(), swipe_right_by_position(), swipe_up_by_position(), and swipe_down_by_position() are enabled according to the sliding type to implement the sliding operation, thereby generating the corresponding test code.

[0064] Scenario three: When the action category of the operation is a long press, the test code corresponding to the operation is generated according to the coordinate data in the step data of the operation and the operation duration; in this embodiment of the present application, when the action category of the operation is a long press, the coordinate information (x, y) and the operation duration time can be obtained from the step data of the operation as the long_press input data of the operation, so as to realize a long press of the coordinates (x, y) on the APP page for a duration of time, and the corresponding test code can be generated.

[0065] Scenario 4: When the action category of the operation is input, the test code corresponding to the operation is generated according to the position coordinates and input content in the step data of the operation. In an embodiment of the present application, when the action category of the operation is input, the position coordinates and input content in the step data can be used to generate the test code corresponding to the operation. In actual applications, TouchActions(driver).tap(x=target_x, y=target_y).perform() can be used to focus on the coordinate (x, y) position, and then driver.set_value(element, text) can be used to implement the input of text information text.

[0066] S105: Acquire a test script of the test case execution step video by combining the test codes corresponding to the operations in order.

[0067] In the embodiment of the present application, the test codes corresponding to each operation can be combined in order to obtain the test script corresponding to the test case execution step video. In practical applications, after each operation is converted into a test script, a sleep time of 1-2s can be added to each operation to reduce operation errors. If the python language is used, the test code can be saved to testcaseName.py. The test script format is shown in Table 4: Table 4

[0068] In the embodiment of the present application, since there is no need to write an automated test script based on the source code and development architecture of the software to be tested during the test script acquisition process, the test script generation efficiency and the accuracy of the test script can be effectively improved.

[0069] In an embodiment of the present application, the action data set of each operation in the test case execution step video can be obtained by analyzing the test case execution step video, and then the action data set of each operation can be identified to obtain the action category of each operation, and then the action data set of each operation, the action category of each operation and the interval time between adjacent operations can be combined to obtain the step data of each operation, and finally the test script of the test case execution step video can be obtained based on the step data of each operation. The process of extracting operation information from the test case execution step video and converting it into a test script is simplified. Not only can the efficiency of obtaining the test script be improved, but also the complexity of converting the video data into the test script is reduced, so that non-professionals can easily generate automated test scripts, which helps to reduce costs.

[0070] Figure 5 The schematic diagram of the structure of the device provided by an exemplary embodiment of the present application is shown. The test script acquisition device can be implemented as follows: Figure 1 In the illustrated embodiment, all or part of the contents, the test script acquisition device includes: a first acquisition module 501 , a second acquisition module 502 , a third acquisition module 503 , a generation module 504 and a fourth acquisition module 505 .

[0071] In an embodiment of the present application, a first acquisition module 501 is used to acquire an action data set of each operation in the test case execution step video by analyzing the test case execution step video, wherein the action data set includes at least one operation data corresponding to the operation; a second acquisition module 502 is used to acquire an action category of each operation by identifying the action data set of each operation, wherein the action category is used to characterize the test step corresponding to the operation; a third acquisition module 503 is used to acquire step data of each operation based on the action data set of each operation and the action category of each operation, wherein the step data is used to characterize the overall action corresponding to the operation; a generation module 504 is used to generate a test code corresponding to each operation based on the step data of each operation; a fourth acquisition module 505 is used to acquire a test script of the test case execution step video by combining the test codes corresponding to each operation in sequence.

[0072] In an optional implementation, the first acquisition module 501, when used to analyze the test case execution step video to obtain the action data set of each operation in the test case execution step video, is specifically used to: Obtaining a video of the test case execution steps in a predetermined format, wherein the video of the test case execution steps includes an operation label and a keyboard label; The image recognition technology is used to analyze the test case execution step video frame by frame to obtain the action data set of each operation.

[0073] In an optional implementation, the operation includes a non-input operation and an input operation; When the first acquisition module 501 is used to analyze the test case execution step video frame by frame by using the image recognition technology to obtain the action data set of each operation, it is specifically used to: When it is recognized that there is an operation label in the first frame and there is no operation label in the second frame, it is determined that the first frame to the second frame is a non-input operation and an action data set of the non-input operation is obtained, wherein the order of the second frame is greater than that of the first frame, and the action data set of the non-input operation includes at least one first operation data, and the format of the first operation data is {A, frame, keyboard_use}, A is the center position of the operation label in the third frame, frame is the frame number of the third frame, when there is a keyboard label in the third frame, keyboard_use is 1, otherwise, keyboard_use is 0, and the third frame is a video frame whose order is greater than or equal to the first frame and less than the second frame.

[0074] In the above optional implementation, when the first acquisition module 501 is used to analyze the test case execution step video frame by frame by using the image recognition technology to obtain the action data set of each operation, it is also used to: When it is identified that the fourth frame has an operation label and a keyboard label, and there is no operation label and keyboard label in the fifth frame, it is determined that the fourth frame to the fifth frame are an input operation and an action data set of the input operation is obtained, wherein the order of the fifth frame is greater than the fourth frame, the action data set of the input operation includes at least one second operation data, and the format of the second operation data is {A, frame, keyboard_use}, A is the center position of the operation label in the sixth frame, frame is the frame number of the sixth frame, when there is a keyboard label in the sixth frame, keyboard_use is 1, otherwise, keyboard_use is 0, and the sixth frame is a video frame whose order is greater than or equal to the fourth frame and less than the fifth frame.

[0075] In an optional implementation, the second acquisition module 502, when used to obtain the action category of each operation by identifying the action data set of each operation, is specifically used to: By performing feature extraction on the action data set of each operation, the target features of each operation are obtained, wherein the target features include: change amount, action duration, keyboard usage status, starting position and ending position; The action category of each operation is obtained by performing action recognition on the target feature of each operation using a trained action category recognition model.

[0076] In an optional implementation, the second acquisition module 502, when used to extract features from the action data set of each operation to obtain the target features of each operation, is specifically used to: Obtaining the change amount of each operation by calculating the coordinate difference between each frame and the previous frame in the action data set of each operation; Obtaining the action duration of each operation by calculating the difference between the frame value of the last frame and the frame value of the first frame in the action data set of each operation; Obtaining the keyboard usage status of each operation through the action data set of each operation; The starting position and the ending position of each operation are obtained through the coordinates of the first frame and the coordinates of the last frame in the action data set of each operation.

[0077] In an optional implementation, the action category recognition model is trained in the following manner: Get training data; Obtaining target training data by preprocessing the training data; Obtaining a feature vector corresponding to the target training data by inputting the target training data into an encoder; By inputting the feature vector into a decoder, an action category corresponding to the feature vector is obtained; The trained action category recognition model is obtained by training the encoder and the decoder.

[0078] In an optional implementation, obtaining a feature vector corresponding to the target training data by inputting the target training data into an encoder includes: Obtaining fusion features by fusing features of the change amount, the starting position, and the ending position in the target training data; The weights of the keyboard usage status and action duration in the target training data are calculated and weighted by using an attention mechanism to obtain weighted keyboard usage status and action duration; By merging the weighted keyboard usage status and action duration and the fusion feature, a merged feature is obtained; The merged features are processed through a fully connected layer to obtain a feature vector corresponding to the target training data.

[0079] In an optional implementation, when the generating module 504 is used to generate the test code corresponding to each of the operations according to the step data of each of the operations, it is specifically used to: According to the action category in the step data of each operation, a test code corresponding to each operation is generated, wherein the action category includes: left swipe, right swipe, up swipe, down swipe, click, long press and input.

[0080] In an optional implementation, when the generating module 504 is used to generate the test code corresponding to each of the operations according to the action category in the step data of each of the operations, it is specifically used to: In the case where the action category of the operation is click, generating a test code corresponding to the operation according to the coordinate data in the step data of each operation; When the action category of the operation is one of left swipe, right swipe, up swipe, and down swipe, generating a test code corresponding to the operation according to the coordinate data in the step data of the operation and the specific sliding direction; In the case where the action category of the operation is long press, generating a test code corresponding to the operation according to the coordinate data in the step data of the operation and the operation duration; When the action category of the operation is input, a test code corresponding to the operation is generated according to the position coordinates and input content in the step data of the operation.

[0081] The test script acquisition device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobilepersonal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0082] The test script acquisition device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0083] The test script acquisition device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented in the illustrated method embodiment will not be described again here.

[0084] Figure 6 The overall architecture diagram of another test script device provided by an exemplary embodiment of the present application is shown. The test script device comprises: a test video data extraction module 601, a video data behavior recognition module 602 and a behavior conversion test script module 603.

[0085] In an embodiment of the present application, the test video data extraction module 601 can analyze the test video files in the test case folder, obtain the operation data in the video, and output the data to the video data behavior recognition module; the video data behavior recognition module 602 can use machine learning technology to train the neural network model to learn according to the training data set of key operation data, realize automatic recognition of the behavior of key data, and output the recognized operation data to the behavior to test script module; the behavior to test script module 603 can convert the test operation behavior recognized by the video data behavior recognition module into a corresponding behavior test script, and combine it with the overall use case framework to form a complete use case script.

[0086] Alternatively, if Figure 7 As shown, an embodiment of the present application also provides an electronic device 700, including a processor 701 and a memory 702, wherein the memory 702 stores programs or instructions that can be executed on the processor 701, and when the program or instructions are executed by the processor 701, the various steps of the method embodiment of the above-mentioned test script acquisition method are implemented, and the same technical effect can be achieved. To avoid repetition, they are not repeated here.

[0087] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0088] Figure 8 FIG. 8 is a block diagram of another electronic device 800 according to an exemplary embodiment of the present application. The electronic device 800 may be implemented as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart watch, a TV, etc. The electronic device 800 may also be called a user device, a portable terminal, a laptop terminal, a desktop terminal, etc.

[0089] Typically, the electronic device 800 includes a processor 801 and a memory 802 .

[0090] The processor 801 may include one or more processing cores, such as a 4-core processor, a 10-core processor, etc. The processor 801 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 801 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 801 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 801 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0091] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one instruction, which is used to be executed by the processor 801 to implement all or part of the steps in the test script acquisition method shown in the present application.

[0092] In some embodiments, the electronic device 800 may further optionally include: a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 803 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807 and a power supply 808.

[0093] In some embodiments, the electronic device 800 further includes one or more sensors 809 , including but not limited to: an acceleration sensor 810 , a gyroscope sensor 811 , a pressure sensor 812 , an optical sensor 813 , and a proximity sensor 814 .

[0094] Those skilled in the art will understand that Figure 8 The structure shown in the figure does not constitute a limitation on the electronic device 800, and may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0095] An embodiment of the present application also provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the method embodiment of the above-mentioned test script acquisition method is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0096] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0097] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the method embodiment of the above-mentioned test script acquisition method, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0098] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0099] An embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the various processes of the above-mentioned test script acquisition method and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0100] It should be noted that, in this article, the term "includes", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device. In the absence of further restrictions, an element defined by the sentence "including one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be noted that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples. Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present application can essentially or the part that contributes to the prior art can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, disk, CD-ROM), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present application.

[0101] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A test script acquisition method, characterized in that: include: By analyzing the test case execution step video, an action data set of each operation in the test case execution step video is obtained, wherein the action data set includes at least one operation data corresponding to the operation; By identifying the action data set of each of the operations, an action category of each of the operations is obtained, wherein the action category is used to characterize the test step corresponding to the operation; According to the action data set of each operation and the action category of each operation, step data of each operation is acquired, wherein the step data is used to characterize the overall action corresponding to the operation; Generate test codes corresponding to each operation according to the step data of each operation; By combining the test codes corresponding to the various operations in sequence, a test script of the test case execution step video is obtained.

2. The method according to claim 1, characterized in that: The step of analyzing the test case execution step video to obtain the action data set of each operation in the test case execution step video includes: Obtaining a video of the test case execution steps in a predetermined format, wherein the video of the test case execution steps includes an operation label and a keyboard label; The image recognition technology is used to analyze the test case execution step video frame by frame to obtain the action data set of each operation.

3. The method according to claim 2, characterized in that The operations include non-input operations and input operations; The step of performing frame-by-frame analysis on the test case execution step video by using image recognition technology to obtain the action data set of each operation includes: When it is recognized that there is an operation label in the first frame and there is no operation label in the second frame, it is determined that the first frame to the second frame is a non-input operation and an action data set of the non-input operation is obtained, wherein the order of the second frame is greater than that of the first frame, and the action data set of the non-input operation includes at least one first operation data, and the format of the first operation data is {A, frame, keyboard_use}, A is the center position of the operation label in the third frame, frame is the frame number of the third frame, when there is a keyboard label in the third frame, keyboard_use is 1, otherwise, keyboard_use is 0, and the third frame is a video frame whose order is greater than or equal to the first frame and less than the second frame.

4. The method according to claim 3, characterized in that The step of analyzing the test case execution step video frame by frame by using image recognition technology to obtain the action data set of each operation also includes: When it is identified that the fourth frame has an operation label and a keyboard label, and there is no operation label and keyboard label in the fifth frame, it is determined that the fourth frame to the fifth frame are an input operation and an action data set of the input operation is obtained, wherein the order of the fifth frame is greater than the fourth frame, the action data set of the input operation includes at least one second operation data, and the format of the second operation data is {A, frame, keyboard_use}, A is the center position of the operation label in the sixth frame, frame is the frame number of the sixth frame, when there is a keyboard label in the sixth frame, keyboard_use is 1, otherwise, keyboard_use is 0, and the sixth frame is a video frame whose order is greater than or equal to the fourth frame and less than the fifth frame.

5. The method according to claim 4, characterized in that The step of identifying the action data set of each operation to obtain the action category of each operation includes: By performing feature extraction on the action data set of each operation, the target feature of each operation is obtained, wherein the target feature includes: change amount, action duration, keyboard usage status, starting position and ending position; The action category of each operation is obtained by performing action recognition on the target feature of each operation using a trained action category recognition model.

6. The method according to claim 5, characterized in that The step of extracting features from action data sets of each operation to obtain target features of each operation includes: Obtaining the change amount of each operation by calculating the coordinate difference between each frame and the previous frame in the action data set of each operation; Obtaining the action duration of each operation by calculating the difference between the frame value of the last frame and the frame value of the first frame in the action data set of each operation; Obtaining the keyboard usage status of each operation through the action data set of each operation; The starting position and the ending position of each operation are obtained through the coordinates of the first frame and the coordinates of the last frame in the action data set of each operation.

7. The method according to claim 5, characterized in that The action category recognition model is trained in the following way: Get training data; Obtaining target training data by preprocessing the training data; Obtaining a feature vector corresponding to the target training data by inputting the target training data into an encoder; By inputting the feature vector into a decoder, an action category corresponding to the feature vector is obtained; The trained action category recognition model is obtained by training the encoder and the decoder.

8. The method according to claim 7, characterized in that The step of obtaining a feature vector corresponding to the target training data by inputting the target training data into an encoder comprises: Obtaining fusion features by fusing features of the change amount, the starting position, and the ending position in the target training data; The weights of the keyboard usage status and action duration in the target training data are calculated and weighted by using an attention mechanism to obtain weighted keyboard usage status and action duration; By merging the weighted keyboard usage status and action duration and the fusion feature, a merged feature is obtained; The merged features are processed through a fully connected layer to obtain a feature vector corresponding to the target training data.

9. The method according to any one of claims 1 to 8, characterized in that: According to the step data of each operation, a test code corresponding to each operation is generated, including: According to the action category in the step data of each operation, a test code corresponding to each operation is generated, wherein the action category includes: left swipe, right swipe, up swipe, down swipe, click, long press and input.

10. The method according to claim 9, characterized in that The step of generating test codes corresponding to each of the operations according to the action categories in the step data of each of the operations comprises: In the case where the action category of the operation is click, generating a test code corresponding to the operation according to the coordinate data in the step data of each operation; When the action category of the operation is one of left swipe, right swipe, up swipe, and down swipe, generating a test code corresponding to the operation according to the coordinate data in the step data of the operation and the specific sliding direction; In the case where the action category of the operation is long press, generating a test code corresponding to the operation according to the coordinate data in the step data of the operation and the operation duration; When the action category of the operation is input, a test code corresponding to the operation is generated according to the position coordinates and input content in the step data of the operation.

11. A test script acquisition device, characterized in that: include: A first acquisition module is used to acquire an action data set of each operation in the test case execution step video by analyzing the test case execution step video, wherein the action data set includes at least one operation data corresponding to the operation; A second acquisition module is used to acquire the action category of each operation by identifying the action data set of each operation, wherein the action category is used to characterize the test step corresponding to the operation; A third acquisition module is used to acquire step data of each operation according to the action data set of each operation and the action category of each operation, wherein the step data is used to represent the overall action corresponding to the operation; A generating module, used for generating a test code corresponding to each of the operations according to the step data of each of the operations; The fourth acquisition module is used to obtain the test script of the test case execution step video by combining the test codes corresponding to the operations in sequence.

12. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the test script acquisition method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the test script acquisition method according to any one of claims 1 to 10 are implemented.

14. A computer program product, characterized in that The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to execute the steps of the test script acquisition method according to any one of claims 1 to 10.

Images