A method, system and product for ui automated testing

Abstract

Embodiments of the present application provide a method, system and product for UI automatic testing, the method comprising: a feature extraction network performing feature extraction on a car machine interface, a starting sliding box and a target region to be tested respectively, obtaining and connecting car machine interface features, sliding box features and target region features to obtain connected features; inputting the connected features into a first reinforcement learning model for processing to obtain position information of the target region in the car machine interface; inputting a current mechanical arm environment state into a second reinforcement learning model for processing to obtain a best motion trajectory of a mechanical arm end stylus moving to the target region, the current mechanical arm environment state including the position information of the target region in the car machine interface; and based on the best motion trajectory, controlling the mechanical arm end stylus to move to the target region for automatic testing. The purpose is to reduce the time complexity of target region recognition and positioning and improve the recognition effect, thereby improving the efficiency and effect of automatic testing.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of UI testing, in particular to a UI automatic testing method, system and product. BACKGROUND

[0002] It is very important to ensure the stability of the interaction between users and the vehicle-mounted machine interface for improving software quality and user satisfaction. The UI automatic testing of the existing vehicle machine interface is an automatic testing technology based on the user interface. It simulates user behaviors such as clicking based on the recognition and positioning of interface elements to verify whether the software interface meets the expectations. It plays an important role in improving testing efficiency, reducing costs, ensuring testing accuracy and consistency, etc. The existing element recognition and positioning methods include a keyword-driven method, a UI control search-based method, and an image matching-based method.

[0003] The image matching-based method can realize the recognition and positioning of the target control by finding the image region in the current interface that is the same as or similar to the given target region picture to be operated. Its use rate gradually increases because it is not easily disturbed by code logic. However, the existing image matching-based method is mostly based on traditional algorithms such as template matching and feature point matching. Such traditional image target matching methods traverse the entire picture until the target region is found, which has high time complexity and poor recognition effect. SUMMARY

[0004] Therefore, the embodiments of the present application provide a UI automatic testing method, system and product. The purpose is to reduce the time complexity of target region recognition and positioning and improve the recognition effect, so as to improve the efficiency and effect of automatic testing.

[0005] The first aspect of the embodiments of the present application provides a UI automatic testing method, which comprises:

[0006] The feature extraction network is used to extract features of the vehicle machine interface, the starting sliding box and the target region to be tested respectively, and obtain the vehicle machine interface feature, the sliding box feature and the target region feature. The sliding box has the same size as the target region.

[0007] The vehicle machine interface feature, the sliding box feature and the target region feature are connected to obtain a connection feature.

[0008] The connection feature is determined as the vehicle machine interface environment state and is input into a first reinforcement learning model for processing to obtain the position information of the target region in the vehicle machine interface.

[0009] inputting a current robot arm environment state into a second reinforcement learning model for processing to obtain an optimal motion trajectory of a robot arm end stylus moving to the target region, the current robot arm environment state including position information of the target region in the vehicle-machine interface and spatial position information of the target region in the vehicle;

[0010] controlling the robot arm end stylus to move to the target region for automatic testing based on the optimal motion trajectory.

[0011] Optionally, a feature extraction network is used to extract features of the vehicle-machine interface, the starting sliding box and the target region to be tested, to obtain vehicle-machine interface features, sliding box features and target region features, including:

[0012] inputting the vehicle-machine interface, the starting sliding box and the target region to be tested into the feature extraction network for feature extraction;

[0013] a spatial pyramid pooling layer of the feature extraction network is used to unify the vehicle-machine interface features, the sliding box features and the target region features to the same scale.

[0014] Optionally, the current robot arm environment state is inputted into the second reinforcement learning model for processing to obtain an optimal motion trajectory of the robot arm end stylus moving to the target region, including:

[0015] determining a current robot arm end stylus position and a robot arm end stylus speed;

[0016] determining a robot arm motion range and environment information, position information of the target region, the current robot arm end stylus position and the robot arm end stylus speed as the current robot arm environment state;

[0017] inputting the current robot arm environment state into the second reinforcement learning model for processing to obtain an optimal robot arm end stylus speed sequence of the robot arm end stylus moving to the target region;

[0018] the controlling the robot arm end stylus to move to the target region for automatic testing based on the optimal motion trajectory, including:

[0019] based on the optimal robot arm end stylus speed sequence, controlling the robot arm end stylus to move to the target region for automatic testing along the optimal robot arm end stylus speed sequence by the robot arm.

[0020] Optionally, the method further includes:

[0021] constructing a first reinforcement learning model;

[0022] inputting a training sample environment state into the first reinforcement learning model for model training, the training sample environment state being an environment state composed of a car machine interface feature, a sliding box feature and a target area feature;

[0023] According to the training sample environment state, the Q value of each action is calculated through the Q value prediction network in the first reinforcement learning model, and the execution action is selected based on the action selection strategy;

[0024] Based on the selected execution action, the next moment training sample environment state of the training sample environment state and the immediate reward are determined;

[0025] The training sample environment state, the execution action, the immediate reward and the next moment training sample environment state are determined as an experience tuple and stored in an experience replay buffer;

[0026] A batch of experience tuples are randomly sampled from the experience replay buffer, the Q value prediction network is trained, the network parameters of the Q value prediction network are updated until the performance of the Q value prediction network is stable, and a trained first reinforcement learning model is obtained.

[0027] Optionally, according to the training sample environment state, the Q value of each action is calculated through the Q value prediction network in the first reinforcement learning model, and the execution action is selected based on the action selection strategy, including:

[0028] According to the training sample environment state, the Q value of each action is calculated through the Q value prediction network algorithm in the first reinforcement learning model;

[0029] The non-preferred action in each action is determined based on a priori rule, and the determined non-preferred action is assigned a Q value less than a set threshold value;

[0030] According to the Q value of each action and the ε-greedy strategy, the execution action is selected from each action.

[0031] Optionally, a batch of experience tuples are randomly sampled from the experience replay buffer, the Q value prediction network is trained, the network parameters of the Q value prediction network are updated until the performance of the Q value prediction network is stable, and a trained first reinforcement learning model is obtained, including:

[0032] A batch of experience tuples are randomly sampled from the experience replay buffer, and the best action in each next moment training sample environment state is selected through the ε-greedy strategy based on the next moment training sample environment state recorded in each experience tuple, the best action being the action corresponding to the maximum Q value obtained by calculating the Q value of each action through the target network in the first reinforcement learning model;

[0033] updating parameters of the Q value prediction network according to an error between a Q value corresponding to the same experience tuple and a target Q value, the target Q value being a Q value of an optimal action;

[0034] copying the parameters of the Q value prediction network to the target network to update the parameters of the target network after every first time length;

[0035] obtaining a first trained reinforcement learning model until performance of the Q value prediction network is stable.

[0036] Optionally, the method further comprises:

[0037] constructing a second reinforcement learning model;

[0038] inputting a training sample environment state to the second reinforcement learning model for model training, the training sample environment state being an environment state composed of a mechanical arm movement range and environment information, position information of a target region, a position of a mechanical arm end touch pen and a speed of the mechanical arm end touch pen;

[0039] determining an execution action through a policy network in the second reinforcement learning model according to the training sample environment state;

[0040] determining a next moment training sample environment state of the training sample environment state and an immediate reward based on the determined execution action;

[0041] determining the training sample environment state, the execution action, the immediate reward and the next moment training sample environment state as an experience tuple and storing the experience tuple to an experience replay buffer;

[0042] randomly sampling a batch of experience tuples from the experience replay buffer, training the policy network and a value network in the second reinforcement learning model, updating network parameters of the policy network and the value network, and obtaining a second trained reinforcement learning model until performance of the second reinforcement learning model is stable.

[0043] Optionally, determining an execution action through a policy network in the second reinforcement learning model according to the training sample environment state comprises:

[0044] determining an initial execution action through the policy network in the second reinforcement learning model according to the training sample environment state;

[0045] adding random noise to the initial execution action to obtain a final execution action.

[0046] Optionally, a batch of experience tuples are randomly sampled from the experience replay buffer, the policy network and the value network in the second reinforcement learning model are trained, network parameters of the policy network and the value network are updated until performance of the second reinforcement learning model is stable, and a trained second reinforcement learning model is obtained, including:

[0047] A batch of experience tuples are randomly sampled from the experience replay buffer, and a Q value of each experience tuple is determined by the value network;

[0048] According to a training sample environment state at a next time in each experience tuple, a target execution action corresponding to each experience tuple is determined by a target policy network in the second reinforcement learning model;

[0049] Based on the determined target execution action corresponding to each experience tuple, a target Q value corresponding to the training sample environment state at the next time and the target execution action in each experience tuple is calculated by the target value network;

[0050] Based on a difference between the Q value and the target Q value corresponding to the same experience tuple, parameters of the value network are updated;

[0051] Based on gradient information of the value network on the action, parameters of the policy network are updated;

[0052] After every second time length, the parameters of the policy network are copied to the target policy network, and the parameters of the value network are copied to the target value network, so as to update the parameters of the target policy network and the target value network;

[0053] Until performance of the policy network and the value network in the second reinforcement learning model is stable, a trained second reinforcement learning model is obtained.

[0054] The second aspect of the embodiments of the application provides a system for UI automatic testing, and the system comprises:

[0055] A feature extraction module is configured to extract features of a car machine interface, a starting sliding box and a target region to be tested by a feature extraction network, so as to obtain car machine interface features, sliding box features and target region features, and the sliding box has the same size as the target region.

[0056] A feature connection module is configured to connect the car machine interface features, the sliding box features and the target region features, so as to obtain connection features.

[0057] A target region identification and positioning module is configured to determine that the connection features are a car machine interface environment state, input the car machine interface environment state into a first reinforcement learning model for processing, and obtain position information of the target region in the car machine interface.

[0058] The motion trajectory determination module is configured to input a current robot arm environment state into the second reinforcement learning model for processing, to obtain an optimal motion trajectory of a robot arm end stylus moving to the target region, the current robot arm environment state including position information of the target region in the vehicle machine interface and spatial position information of the target region in the vehicle.

[0059] The test module is configured to control the robot arm end stylus to move to the target region for automatic testing based on the optimal motion trajectory.

[0060] The third aspect of the embodiments of the present application provides an electronic device, which comprises a processor, a memory, and a computer program stored in the memory and running on the processor, and the computer program implements the steps in the method for UI automatic testing according to the first aspect of the present application when executed by the processor.

[0061] The fourth aspect of the embodiments of the present application provides a computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, and the computer program implements the steps in the method for UI automatic testing according to the first aspect of the present application when executed by a processor.

[0062] The method for UI automatic testing provided by the present application has the following advantages:

[0063] The method for automatically testing a UI provided by the embodiment of the application first extracts features of a car-machine interface, a starting sliding box and a target region to be tested through a feature extraction network, to obtain car-machine interface features, sliding box features and target region features, the sliding box and the target region being of the same size; the car-machine interface features, the sliding box features and the target region features are connected to obtain connected features; the connected features are determined as a car-machine interface environment state and input to a first reinforcement learning model for processing to obtain position information of the target region in the car-machine interface; a current mechanical arm environment state is input to a second reinforcement learning model for processing to obtain a best motion trajectory of a mechanical arm end touch pen moving to the target region, the current mechanical arm environment state including the position information of the target region in the car-machine interface and spatial position information of the target region in the car; based on the best motion trajectory, the mechanical arm end touch pen is controlled to move to the target region for automatic testing. Thus, the embodiment introduces a reinforcement learning model, inputs a current car-machine interface environment state to the reinforcement learning model to determine a target region to be operated, without traversing the whole image, so that the time complexity can be reduced, and with the improvement of the quality of sample data of the trained model, the recognition effect of this method can also be greatly improved. Meanwhile, the reinforcement learning model is also used to determine the best motion trajectory of the mechanical arm end touch pen moving from a current position to the position of the target region, so that the test efficiency of moving the mechanical arm end touch pen to the target position for automatic testing can be improved, and the recognition effect of the position recognition and positioning of the target region is improved, and the test effect of the automatic testing is also improved. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the application, the drawings needed to be used in the description of the embodiments of the application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0065] Figure 1 A flowchart of a method for automatically testing a UI according to an embodiment of the application is shown.

[0066] Figure 2 A car-machine interface diagram in a method for automatically testing a UI according to an embodiment of the application is shown.

[0067] Figure 3 A sliding box sliding diagram in a method for automatically testing a UI according to an embodiment of the application is shown.

[0068] Figure 4A structure diagram of a first reinforcement learning model in a method of UI automatic testing shown in an embodiment of the present application;

[0069] Figure 5 Another flowchart of a method of UI automatic testing shown in an embodiment of the present application;

[0070] Figure 6 A schematic diagram of a system of UI automatic testing shown in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0072] Reference Figure 1 , Figure 1 A flowchart of a method of UI automatic testing shown in an embodiment of the present application. As shown in FIG. 1, the method comprises: Figure 1

[0073] Step S1: performing feature extraction on the car machine interface, the starting sliding box and the target region to be tested respectively through a feature extraction network to obtain car machine interface features, sliding box features and target region features, the sliding box being the same size as the target region.

[0074] In the embodiment, when the UI interface of the car machine is automatically tested, there are corresponding test cases according to different test contents, and the image data of the target region of the UI interface to be tested is recorded in the test case. Each test case starts with the same initial interface, and each time a new interface is entered, the retrieval starts from the specified starting point L0, and the starting time of the sliding box is the image data generated at the specified starting point L0 which is the same size as the image of the target region, as shown in FIG. 2. Figure 2 Based on the test case, the image data of the target region corresponding to the test case is determined. Then, the feature extraction network in the first reinforcement learning model for determining the specific position of the target region in the car machine interface is used to perform feature extraction on the image data of the target region, the image data of the car machine interface in the starting state and the image data of the sliding box in the starting state, respectively, to obtain the target region features corresponding to the target region, the car machine interface features corresponding to the car machine interface and the sliding box features corresponding to the sliding box.

[0075] Step S2: performing feature connection on the car machine interface features, the sliding box features and the target region features to obtain connection features. ​

[0076] In this embodiment, the vehicle interface features, sliding box features, and target area features obtained in step S1 are connected to obtain the connected features.

[0077] Step S3: Determine that the connection feature is the vehicle interface environment state, input it into the first reinforcement learning model for processing, and obtain the location information of the target area in the vehicle interface.

[0078] In this embodiment, after obtaining the corresponding connection feature through step S2, the connection feature is used as the current vehicle interface environment state. In this vehicle interface environment state, the vehicle interface is the environment, the slider is the agent performing the action, and the target area feature is the position that the agent (i.e., the slider) needs to reach by performing the corresponding action, which is the target. The actions performed by the agent include four actions: up, down, left, and right. The preferred movement range for each movement is 1% of the image data size of the vehicle interface. Figure 3 As shown, the vehicle interface environment state is then input into the first reinforcement learning model for processing. Based on this environment state, the first reinforcement learning model obtains the action sequence A of the agent (i.e., the slider) moving to the target area within the environment (i.e., the vehicle interface). Based on this action sequence A and the position information of the slider's initial point L0, the position information of the target area within the vehicle interface is determined to be L0+A. The first reinforcement learning model is preferably the DQN (Deep Q-Network) reinforcement learning algorithm. It should be understood that the first reinforcement learning model can also be other reinforcement learning algorithms, which are not specifically limited here.

[0079] Step S4: Input the current robotic arm environment state into the second reinforcement learning model for processing to obtain the optimal motion trajectory of the stylus at the end of the robotic arm to the target area. The current robotic arm environment state includes the position information of the target area in the vehicle interface and the spatial position information of the target area inside the vehicle.

[0080] In the embodiment, after obtaining the position information of the target region in the vehicle-machine interface through step S3, the current mechanical arm environment state is determined, which at least includes the position information of the target region in the vehicle-machine interface and the spatial position information of the target region in the vehicle to which the mechanical arm end touch pen needs to move. The determined current mechanical arm environment state is then input to the second reinforcement learning model for processing. The second reinforcement learning model will determine the optimal motion trajectory of the mechanical arm end touch pen moving from its current position to the position of the target region based on the current mechanical arm environment state. The second reinforcement learning model is preferably a Deep Deterministic Policy Gradient (DDPG) algorithm based on reinforcement learning. It should be understood that the second reinforcement learning model can also be other reinforcement learning algorithms, which are not specifically limited here.

[0081] Step S5: based on the optimal motion trajectory, controlling the end touch pen of the mechanical arm to move to the target region for automatic testing.

[0082] In the embodiment, after obtaining the optimal motion trajectory of the mechanical arm end touch pen moving from its current position to the position of the target region through step S4, the end touch pen of the mechanical arm is controlled to move to the position of the target region along the motion trajectory based on the optimal motion trajectory, so as to automatically test the vehicle-machine interface, such as click test, slide test, etc.

[0083] The method for automatically testing a UI provided by the embodiments of the present application first extracts features of a car-machine interface, a starting sliding box and a target region to be tested through a feature extraction network, to obtain car-machine interface features, sliding box features and target region features, the sliding box and the target region having the same size; the car-machine interface features, the sliding box features and the target region features are connected to obtain connected features; the connected features are determined as a car-machine interface environment state and input to a first reinforcement learning model for processing to obtain position information of the target region in the car-machine interface; a current mechanical arm environment state is input to a second reinforcement learning model for processing to obtain a motion trajectory of a stylus at the end of the mechanical arm moving to the target region; and the motion trajectory is used to control the mechanical arm to move to the target region for automatic testing. Thus, the embodiments of the present application introduce a reinforcement learning model, input the current car-machine interface environment state to the reinforcement learning model to determine the target region to be operated, without traversing the entire image, so as to reduce the time complexity, and with the improvement of the quality of sample data of the trained model, the recognition effect of this method can also be greatly improved. Meanwhile, the reinforcement learning model is also used to determine the best motion trajectory of the stylus at the end of the mechanical arm moving from the current position to the position of the target region, so as to improve the test efficiency of the stylus at the end of the mechanical arm moving to the target position for automatic testing, and with the improvement of the recognition effect of the position recognition and positioning of the target region, the test effect of the automatic testing is also improved.

[0084] In combination with the above embodiments, in an implementation, the embodiments of the present application also provide a method for automatically testing a UI. In the method for automatically testing a UI, step S1 can include: inputting a car-machine interface, a starting sliding box and a target region to be tested to a feature extraction network for feature extraction; and using a spatial pyramid pooling layer of the feature extraction network to unify car-machine interface features, sliding box features and target region features to the same scale.

[0085] In the embodiments, since the size of the image data of the car-machine interface is inconsistent with the sizes of the image data of the target region and the sliding box, the feature extraction network in the first reinforcement learning model is adjusted to make the adjusted feature extraction network output the same scale of results for input images of different scales, for the convenience of subsequent full connection layer network input. Specifically, a spatial pyramid pooling (SPP) layer is arranged after the last pooling layer of the feature extraction network in the first reinforcement learning model, to unify the scale of the output features. The network structure of the feature extraction network is preferably five convolution layers + a Relu layer + a pooling layer + one SPP layer, as shown in Figure 4As shown, the image data of the car-machine interface, the image data of the starting sliding box, and the image data of the target region to be tested are respectively input to the feature extraction network for feature extraction. The features are extracted by all the layers in front of the SPP layer, and the corresponding features are obtained. Then, the last SPP layer unifies the feature scales to obtain the car-machine interface features, the sliding box features, and the target region features with the same scale.

[0086] In combination with the above embodiments, in an implementation, the embodiments of the present application also provide a UI automatic testing method. In the UI automatic testing method, step S4 can include steps S41 to S43:

[0087] Step S41: determining the current position of the mechanical arm end touch pen and the current speed of the mechanical arm end touch pen.

[0088] In the present embodiment, the current mechanical arm environment state input to the second reinforcement learning model for processing includes information such as the current position of the mechanical arm end touch pen, the current speed of the mechanical arm end touch pen, the motion range of the mechanical arm, environment information, and the position information of the target region. The mechanical arm can be fixed in the car, and the UI interface test can be performed in the fixed operation space. A silicon touch pen is fixed at the end of the mechanical arm to prevent excessive force from damaging the screen. The current position of the mechanical arm end touch pen refers to the spatial position of the tip of the silicon touch pen at the end of the mechanical arm; the current speed of the mechanical arm end touch pen refers to the spatial vector from the spatial position of the tip of the silicon touch pen at the end of the mechanical arm at the previous moment to the spatial position of the tip of the silicon touch pen at the end of the mechanical arm at the current moment, such as the spatial position of the tip of the silicon touch pen at the end of the mechanical arm at the previous moment (a1, b1, c1) and the spatial position of the tip of the silicon touch pen at the end of the mechanical arm at the current moment (a2, b2, c2). The current speed of the mechanical arm end touch pen refers to the spatial vector from the spatial position (a1, b1, c1) to the spatial position (a2, b2, c2), which includes the motion length and the motion direction. The motion length is the length between the two spatial positions, and the motion direction is the direction from the spatial position (a1, b1, c1) to the spatial position (a2, b2, c2); the motion range of the mechanical arm and the environment information refer to the information of the spatial range that the mechanical arm can move after being fixed in the car. The spatial range can be a spatial sphere, a spatial cuboid, or a spatial cube, which is not limited here; and the position information of the target region is the position information of the region to be tested determined by the first reinforcement learning model. Although this information belongs to the position information of the target region relative to the car-machine interface, the mechanical arm is fixed in the car. Based on the relative position relationship between the mechanical arm and the car-machine interface and the position information of the target region, the mechanical arm can be positioned to the position information of the target region relative to itself.

[0089] In the embodiment, the current position and speed of the mechanical arm end touch pen are determined based on the current state of the mechanical arm and the state at the previous moment.

[0090] In step S42, the motion range and environment information of the mechanical arm, the position information of the target region, the current position and speed of the mechanical arm end touch pen are determined as the current mechanical arm environment state.

[0091] In the embodiment, the motion range and environment information of the mechanical arm, the position information of the target region, the current position and speed of the mechanical arm end touch pen are determined as the current mechanical arm environment state. In the mechanical arm environment state, the mechanical arm is an agent, the speed of the mechanical arm end touch pen is an action of the agent, the motion range and environment information of the mechanical arm is the environment, and the position information of the target region is the position that the agent needs to reach by performing the action, that is, the target. The motion of each motion node of the mechanical arm drives the motion of the mechanical arm end touch pen in the three-dimensional space.

[0092] In step S43, the current mechanical arm environment state is input into the second reinforcement learning model for processing to obtain the optimal mechanical arm end touch pen speed sequence for moving the mechanical arm end touch pen to the target region.

[0093] In the embodiment, the current mechanical arm environment state is input into the second reinforcement learning model. Based on the current mechanical arm environment state, the second reinforcement learning model obtains the optimal sequence of actions (that is, the mechanical arm end touch pen speed sequence) of the agent (that is, the mechanical arm) in the environment (that is, the motion range and environment information of the mechanical arm) from the current state to the position of the target region. The optimal sequence of actions is the optimal mechanical arm end touch pen speed sequence for moving the mechanical arm end touch pen from the current state to the target region.

[0094] In the application, when step S4 includes steps S41 to S43, step S5 can include: based on the optimal mechanical arm end touch pen speed sequence, controlling the mechanical arm end touch pen to move to the target region along the optimal mechanical arm end touch pen speed sequence by the mechanical arm for automatic testing.

[0095] In the embodiment, after obtaining the optimal mechanical arm end touch pen speed sequence by the second reinforcement learning model, based on the optimal mechanical arm end touch pen speed sequence, the mechanical arm end touch pen is controlled to move to the target region along the optimal mechanical arm end touch pen speed sequence by the mechanical arm for automatic testing.

[0096] In combination with the above embodiments, in an implementation, the embodiments of the present application further provide a method for UI automatic testing. In the method for UI automatic testing, the method further comprises steps S01 to S06:

[0097] Step S01: constructing a first reinforcement learning model.

[0098] In the present embodiment, in the actual application process, the first reinforcement learning model for determining the position information of the target region in the car machine interface is a qualified training reinforcement learning model, and before its specific application, the present application will first train a qualified first reinforcement learning model. Specifically, an initial untrained first reinforcement learning model is constructed, and the first reinforcement learning model is a DQN (Deep Q-Network) reinforcement learning algorithm.

[0099] Step S02: inputting a training sample environment state into the first reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of car machine interface features, sliding box features and target region features.

[0100] In the present embodiment, the training sample environment state is inputted into the constructed first reinforcement learning model for model training, and the inputted training sample environment state is also an environment state including car machine interface features, sliding box features and target region features. At the same time, since the sample data used for training the model is large in quantity, there are various image data of car machine interfaces in the image data of the car machine interface corresponding to the car machine interface features in the large amount of training sample environment states, there are various image data of sliding boxes in the image data of the sliding box corresponding to the sliding box features, and there are various image data of target regions in the image data of the target region corresponding to the target region features, so that the samples of the training model are diverse.

[0101] Step S03: calculating the Q value of each action through the Q value prediction network in the first reinforcement learning model according to the training sample environment state, and selecting an executed action based on an action selection strategy.

[0102] In the present embodiment, for the training sample environment state inputted into the constructed first reinforcement learning model, the next step action to be executed in the current training sample environment state is selected, and the selection manner of the step action is as follows: the Q values of the up, down, left and right actions in the step are calculated through the Q value prediction network in the first reinforcement learning model, and then the action to be executed in the step is selected based on the action selection strategy related to the action Q values. For example, there are four actions of up, down, left and right in the step, and finally the down action is determined to be executed in the step based on the Q values corresponding to the four actions and the action selection strategy. The moving range of the sliding box in each step is preferably one percent of the size of the image data of the car machine interface.

[0103] Step S04: determining a training sample environment state at a next time of the training sample environment state and an immediate reward based on the selected execution action.

[0104] In this embodiment, based on the execution action selected in step S03, the training sample environment state after the execution of the execution action selected in step S03 and the obtained immediate reward are determined. The training sample environment state after the execution of the execution action selected in step S03 also includes the car machine interface feature, the sliding box feature and the target area feature. Since the car machine interface and the target area do not change, the car machine interface feature and the target area feature in the training sample environment state after the execution of the execution action selected in step S03 do not change, and only the sliding box feature changes. At the same time, the corresponding immediate reward after the execution of the execution action selected in step S03 is calculated as the similarity between the sliding box feature after the execution of the execution action selected in step S03 and the target area feature to determine the immediate reward. The higher the similarity, the higher the reward, and the lower the similarity, the lower the reward. At the same time, the training sample environment state after the execution of the execution action selected in step S03 and the obtained immediate reward are the training sample environment state at a next time and the immediate reward under the training sample environment state currently processed by the constructed first reinforcement learning model.

[0105] Step S05: determining the training sample environment state, the execution action, the immediate reward and the training sample environment state at a next time as an experience tuple and storing it into an experience replay buffer.

[0106] In this embodiment, after obtaining the training sample environment state at a next time and the immediate reward under the training sample environment state currently processed by the constructed first reinforcement learning model through step S04, the training sample environment state currently processed by the constructed first reinforcement learning model, the execution action determined through step S03, the training sample environment state at a next time and the immediate reward are determined as an experience tuple and stored into an experience replay buffer.

[0107] In the embodiment, since the sliding box slides from the starting position, a plurality of steps of actions are performed before sliding to the target area. Therefore, after the first reinforcement learning model determines the next training sample environment state and the immediate reward after performing the action and the next moment based on the current training sample environment state, the next training sample environment state is determined as the current training sample environment state, and then the first reinforcement learning model determines the next training sample environment state and the immediate reward after performing the action and the next moment based on the new current training sample environment state. In this way, a large number of experience tuples can be obtained for one training sample environment state input to the first reinforcement learning model. Meanwhile, a large number of different training sample environments are input to the first reinforcement learning model, and a larger number of experience tuples are obtained. Therefore, the experience replay buffer pool includes a large number of experience tuples.

[0108] Step S06: randomly sampling a batch of experience tuples from the experience replay buffer pool, training the Q value prediction network, updating the network parameters of the Q value prediction network until the performance of the Q value prediction network is stable, and obtaining a trained first reinforcement learning model.

[0109] In the embodiment, the experience replay buffer pool including a large number of experience tuples is obtained through repeated execution of the above steps S01 to S05. At this time, a batch of experience tuples are randomly sampled from the experience replay buffer pool to train the Q value prediction network in the first reinforcement learning model to update the network parameters of the Q value prediction network. After updating the network parameters of the Q value prediction network, it is determined whether the performance of the Q value prediction network is stable. In the case where the performance of the Q value prediction network is not stable, a batch of experience tuples are randomly sampled from the experience replay buffer pool to train the Q value prediction network in the first reinforcement learning model to update the network parameters of the Q value prediction network. Until the performance of the Q value prediction network is stable, a trained first reinforcement learning model is obtained.

[0110] In combination with the above embodiments, in an implementation, the embodiments of the present application further provide a UI automatic testing method. In the UI automatic testing method, step S03 can include steps S031 to S032:

[0111] Step S031: calculating the Q value of each action through the Q value prediction network algorithm in the first reinforcement learning model according to the training sample environment state.

[0112] In the embodiment, for the training sample environment state input into the constructed first reinforcement learning model, the next step action to be performed in the current training sample environment state is selected, at this time, firstly, the Q value prediction network algorithm in the constructed first reinforcement learning model is used to calculate the Q value of each of the four actions of up, down, left and right in the next step action based on the current training sample environment state.

[0113] Step S032: determining a non-preferred action in each action based on the priori rule, and assigning a Q value less than a set threshold value to the determined non-preferred action; selecting an execution action from each action according to the Q value of each action and the ε-greedy strategy.

[0114] In the embodiment, after the Q value of each of the four actions of up, down, left and right in the next step action is calculated based on step S031, the action that is not expected to be taken is shielded based on the priori rule, that is, a great punishment is given to the action that will cause the agent (that is, the sliding box) to exceed the car interface region, so that the agent avoids selecting the action, and the action that will cause the agent (that is, the sliding box) to exceed the car interface region is the non-preferred action, the Q value calculated before is discarded for the non-preferred action, and a Q value less than a set threshold value is assigned to it, which is a particularly small Q value, so that when the action is selected and executed using the ε-greedy strategy, the non-preferred action will not be selected. Then, the execution action is selected from each action based on the obtained Q value of each action through the ε-greedy strategy.

[0115] In combination with the above embodiments, in an implementation, the embodiment of the application further provides a UI automatic testing method. In the UI automatic testing method, step S06 can include steps S061 to S064:

[0116] Step S061: randomly sampling a batch of experience tuples from the experience replay buffer, and selecting the best action in each next time training sample environment state recorded in each experience tuple through the ε-greedy strategy, the best action being the action corresponding to the maximum Q value obtained by calculating the Q value of each action through the target network in the first reinforcement learning model.

[0117] In the embodiment, the experience replay buffer pool including a large number of experience tuples is obtained through the continuous repeated execution of the above steps S01 to S05, a batch of experience tuples is randomly sampled from the experience replay buffer pool at this time, the training sample environment state of the next moment after the corresponding action is recorded for each experience tuple in the batch of experience tuples, the Q value of each action in the next moment training sample environment state is calculated through the target network in the first reinforcement learning model for each next moment training sample environment state, and then the action with the maximum Q value in the next moment training sample environment state is selected as the optimal action in the next moment training sample environment state through the ε-greedy strategy.

[0118] Step S062: updating the parameters of the Q value prediction network according to the error between the Q value corresponding to the same experience tuple and the target Q value.

[0119] In the embodiment, after obtaining the target Q value under each optimal action through step S061, the network parameters of the Q value prediction network are updated based on the error between the Q value (which is the Q value corresponding to the executed action in the same experience tuple) and the target Q value (which is the Q value corresponding to the optimal action in the next moment training sample environment state based on the same experience tuple) belonging to the same experience tuple.

[0120] Step S063: copying the parameters of the Q value prediction network to the target network to update the parameters of the target network after every first time length.

[0121] In the embodiment, to improve the stability and effect of training, during the continuous updating of the network parameters of the Q value prediction network, the latest network parameters of the Q value prediction network are copied to the target network after every first time length to update the network parameters of the target network. The first time length can be set according to the actual application scenario, which is not specifically limited here.

[0122] Step S064: obtaining the trained first reinforcement learning model until the performance of the Q value prediction network is stable.

[0123] In the embodiment, after training the Q value prediction network with a batch of randomly sampled experience tuples, the Q value prediction network is trained with a batch of randomly sampled experience tuples again until the performance of the trained Q value prediction network tends to be stable or until the first preset number of times of training, and then the trained first reinforcement learning model is determined to be obtained, which can be used to determine the position information of the target region in the car machine interface based on the current car machine interface environment state. The first preset number of times can be set according to the actual application scenario, which is not specifically limited here.

[0124] In combination with the above embodiments, in an implementation, the embodiments of the present application further provide a method for UI automatic testing. In the method for UI automatic testing, the method further comprises steps S001 to S006:

[0125] Step S001: constructing a second reinforcement learning model.

[0126] In the present embodiment, in actual application, the second reinforcement learning model for determining the optimal mechanical arm end stylus speed sequence of the mechanical arm end stylus is a qualified training reinforcement learning model, and before its specific application, the present application will first train to obtain a qualified training second reinforcement learning model. Specifically, an initial untrained second reinforcement learning model is constructed, and the second reinforcement learning model is a deep deterministic policy gradient algorithm (DDPG) based on reinforcement learning.

[0127] Step S002: inputting a training sample environment state into the second reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of mechanical arm movement range and environment information, position information of a target region, mechanical arm end stylus position, and mechanical arm end stylus speed.

[0128] In the present embodiment, the training sample environment state is inputted into the constructed second reinforcement learning model for model training, and the inputted training sample environment state is also an environment state including mechanical arm movement range and environment information, position information of a target region, mechanical arm end stylus position, and mechanical arm end stylus speed. At the same time, since the sample data used for training the model is large in quantity, the position information of the target region in a large number of training sample environment states is different, the mechanical arm end stylus position is different, and the mechanical arm end stylus speed is different, so that the samples for training the model are diverse.

[0129] Step S003: determining an execution action through a policy network in the second reinforcement learning model according to the training sample environment state.

[0130] In the present embodiment, for the training sample environment state inputted into the constructed second reinforcement learning model, the next step action to be executed in the current training sample environment state is selected, and the selection manner of the step action is: determining an execution action through a policy network in the second reinforcement learning model.

[0131] Step S004: determining a next-time training sample environment state of the training sample environment state and an immediate reward based on the determined execution action.

[0132] In the embodiment, based on the execution action determined in step S003, the state of the training sample environment after the execution action selected in step S003 is executed and the instant reward obtained are determined. The state of the training sample environment after the execution action selected in step S003 is executed also includes the mechanical arm movement range and environment information, the position information of the target region, the position of the mechanical arm end touch pen, and the speed of the mechanical arm end touch pen. Since the mechanical arm movement range and environment information and the position information of the target region do not change, the mechanical arm movement range and environment information and the position information of the target region in the state of the training sample environment after the execution action selected in step S003 is executed do not change, and only the position of the mechanical arm end touch pen and the speed of the mechanical arm end touch pen change. At the same time, the instant reward corresponding to the execution action selected in step S003 is calculated as follows: when the mechanical arm end touch pen moves to the position of the target region, the reward r t = 10, otherwise the reward r t = |the position of the mechanical arm end touch pen - the position of the target region| 2 If the mechanical arm end touch pen exceeds the mechanical arm movement range, an additional penalty r0 = -5 is added. At the same time, the state of the training sample environment after the execution action selected in step S003 is executed and the instant reward obtained are the next moment training sample environment state and instant reward of the training sample environment state currently processed by the constructed second reinforcement learning model.

[0133] Step S005: Determine the training sample environment state, the execution action, the instant reward, and the next moment training sample environment state as an experience tuple and store it in the experience replay buffer.

[0134] In the embodiment, after obtaining the next moment training sample environment state and instant reward of the training sample environment state currently processed by the constructed second reinforcement learning model through step S004, the training sample environment state currently processed by the constructed second reinforcement learning model, the execution action determined through step S003, the next moment training sample environment state, and the instant reward are determined as an experience tuple and stored in the experience replay buffer.

[0135] In the embodiment, since the mechanical arm end touch pen moves in the three-dimensional space from the starting position, a plurality of steps of actions are performed before moving to the position where the target region is located. Therefore, after the constructed second reinforcement learning model determines, based on the current training sample environment state, the training sample environment state at the next moment, the immediate reward after the initial action is performed, and the training sample environment state at the next moment is determined as the current training sample environment state, the constructed second reinforcement learning model determines, based on the new current training sample environment state, the training sample environment state at the next moment, the immediate reward after the initial action is performed, and the training sample environment state at the next moment is determined as the current training sample environment state. In this way, for one training sample environment state input to the constructed second reinforcement learning model, a large number of experience tuples can be obtained. Meanwhile, a large number of different training sample environments are input to the constructed second reinforcement learning model, and a larger number of experience tuples are obtained. Therefore, the experience replay buffer pool includes a large number of experience tuples.

[0136] Step S006: randomly sampling a batch of experience tuples from the experience replay buffer pool, training the policy network and the value network in the second reinforcement learning model, updating the network parameters of the policy network and the value network, until the performance of the second reinforcement learning model is stable, and obtaining a trained second reinforcement learning model.

[0137] In the embodiment, the experience replay buffer pool including a large number of experience tuples is obtained through the repeated execution of the above steps S001 to S005. At this time, a batch of experience tuples are randomly sampled from the experience replay buffer pool to train the policy network and the value network in the constructed second reinforcement learning model, so as to update the network parameters of the policy network and the value network. After updating the network parameters of the policy network and the value network, it is determined whether the performance of the policy network and the value network is stable. In the case where the performance of the policy network and the value network is not stable, a batch of experience tuples are randomly sampled from the experience replay buffer pool to train the policy network and the value network in the constructed second reinforcement learning model, so as to update the network parameters of the policy network and the value network. Until the performance of the policy network and the value network is stable, a trained second reinforcement learning model is obtained.

[0138] In combination with the above embodiments, in an implementation, the embodiments of the present application also provide a UI automatic testing method. In the UI automatic testing method, step S003 can include steps S0031 to S0032:

[0139] Step S0031: determining an initial action by a policy network in a second reinforcement learning model according to the training sample environment state.

[0140] In the embodiment, in order to increase the exploration degree of the mechanical arm in the motion range, the application adds noise to the determined execution action. Specifically, for the training sample environment state input into the constructed second reinforcement learning model, the next action to be executed in the current training sample environment state is selected, and the selection manner of the action is: the initial execution action is obtained by determining the policy network in the second reinforcement learning model.

[0141] Step S0032: random noise is added to the initial execution action to obtain a final execution action.

[0142] In the embodiment, random noise is added to the initial execution action determined by the policy network to obtain a final execution action.

[0143] In combination with the above embodiments, in an implementation manner, the embodiment of the application further provides a UI automatic testing method. In the UI automatic testing method, step S006 can include steps S0061 to S0067.

[0144] Step S0061: a batch of experience tuples are randomly sampled from the experience replay buffer, and the Q value of each experience tuple is determined by the value network.

[0145] In the embodiment, the experience replay buffer including a large number of experience tuples is obtained through repeated execution of the above steps S001 to S005, at this time, a batch of experience tuples are randomly sampled from the experience replay buffer, and for each experience tuple in the batch of experience tuples, the execution action and the training sample environment state are recorded, at this time, the Q value of the execution action recorded in each experience tuple is determined by the value network.

[0146] Step S0062: according to the training sample environment state at the next time in each experience tuple, the target execution action corresponding to each experience tuple is determined by the target policy network in the second reinforcement learning model.

[0147] In the embodiment, then, for each next time training sample environment state recorded in each experience tuple in the batch of experience tuples, the execution action in each next time training sample environment state is determined by the target policy network in the constructed second reinforcement learning model, and the execution action is the target execution action.

[0148] Step S0063: based on the determined target execution action corresponding to each experience tuple, the target Q value corresponding to the next time training sample environment state and the target execution action in each experience tuple is calculated by the target value network.

[0149] In this embodiment, for each target execution action in the next time of the training sample environment state obtained through step S0062, the target Q value corresponding to the next time of the training sample environment state and the target execution action of each experience tuple is respectively calculated by the target value network in the constructed second reinforcement learning model.

[0150] Step S0064: Based on the difference between the Q value and the target Q value corresponding to the same experience tuple, the parameters of the value network are updated.

[0151] In this embodiment, after obtaining the target Q value under each target action through step S0063, the network parameters of the value network are updated based on the error between the Q value (which is the Q value corresponding to the execution action recorded in the same experience tuple) and the target Q value (which is the Q value calculated based on the target action in the next time of the training sample environment state) belonging to the same experience tuple.

[0152] Step S0065: Based on the gradient information of the action of the value network, the parameters of the policy network are updated.

[0153] In this embodiment, after updating the parameters of the value network each time through step S0064, the network parameters of the policy network are synchronously updated based on the gradient information of the action of the value network.

[0154] Step S0066: After every second time length, the parameters of the policy network are copied to the target policy network, and the parameters of the value network are copied to the target value network to update the parameters of the target policy network and the target value network.

[0155] In this embodiment, to improve the stability and effect of training, during the process of constantly updating the network parameters of the value network and the policy network, after every second time length, the latest network parameters of the value network are given to the target value network to update the network parameters of the target value network, and the latest network parameters of the policy network are given to the target policy network to update the network parameters of the target policy network. The second time length can be set according to the actual application scenario, which is not specifically limited here.

[0156] Step S0067: Until the performance of the policy network and the value network in the second reinforcement learning model is stable, a trained second reinforcement learning model is obtained.

[0157] In this embodiment, after training the value network and policy network with a randomly sampled batch of experience tuples, another randomly sampled batch of experience tuples is used to train the value network and policy network again. This process continues until the performance of the trained value network and policy network stabilizes or until a second preset number of training iterations is completed. At this point, a qualified second reinforcement learning model is determined. This qualified second reinforcement learning model can then be used to determine the optimal stylus velocity sequence at the end effector of the robotic arm based on the current robotic arm environment. The qualified second reinforcement learning model primarily utilizes the policy network. The second preset number of training iterations can be set according to the actual application scenario and is not specifically limited here.

[0158] In this embodiment, the specific application after the first reinforcement learning model and the second reinforcement learning model have been successfully trained is as follows: Figure 5 As shown, firstly, after selecting the image data of the target area in the vehicle interface based on the test case, an image data of a sliding box with the same size as the image data of the target area is generated at the starting point L0 of the vehicle interface; secondly, the image data of the vehicle interface, the sliding box and the target area are input into the feature extraction network for feature extraction, with the vehicle interface set as the environment, the sliding box set as the agent and the target area set as the target. The next movement action of the agent is obtained by inputting the extracted features of the three elements as state inputs into the first reinforcement learning model. The optimal path to the target area is found through interaction with the environment. Based on this path and the initial position of the slider, the location of the target area in the vehicle interface can be quickly determined. Then, the obtained target area location information, robotic arm movement range and environmental information, current position and speed of the robotic arm's end effector are input as environmental state inputs into the second reinforcement learning model. Through interaction with the environment, the optimal control trajectory for the robotic arm's end effector to move from its initial position to the target area is determined. Based on this optimal control trajectory, the robotic arm controls the end effector to move to the target area for UI interface testing. Finally, after the robotic arm's end effector reaches the target area and completes the test steps, assertions are made between the current vehicle interface and the desired vehicle interface, and a test report is generated. If the assertion fails, the vehicle log is automatically retrieved and stored before executing the next test case; if the assertion succeeds, the next test case is executed directly.

[0159] Based on the same inventive concept, one embodiment of this application provides a system for automatic UI testing, such as... Figure 6 As shown, the system 600 includes:

[0160] The feature extraction module 601 is configured to perform feature extraction on the car-machine interface, the starting sliding box and the target region to be tested respectively by using a feature extraction network, to obtain car-machine interface features, sliding box features and target region features, and the sliding box has the same size as the target region.

[0161] The feature connection module 602 is configured to perform feature connection on the car-machine interface features, the sliding box features and the target region features to obtain connection features.

[0162] The target region identification and positioning module 603 is configured to determine that the connection features are car-machine interface environment states, and input the connection features into a first reinforcement learning model for processing to obtain position information of the target region in the car-machine interface.

[0163] The motion trajectory determination module 604 is configured to input a current robot arm environment state into a second reinforcement learning model for processing to obtain a best motion trajectory of a robot arm end touch pen moving to the target region, and the current robot arm environment state includes the position information of the target region in the car-machine interface and spatial position information of the target region in the car.

[0164] The test module 605 is configured to control the robot arm end touch pen to move to the target region for automatic testing based on the best motion trajectory.

[0165] Optionally, the feature extraction module 601 includes:

[0166] The first feature extraction module is configured to input the car-machine interface, the starting sliding box and the target region to be tested into the feature extraction network for feature extraction.

[0167] The second feature extraction module is configured to uniformly process the car-machine interface features, the sliding box features and the target region features to the same scale by using a spatial pyramid pooling layer of the feature extraction network.

[0168] Optionally, the motion trajectory determination module 604 includes:

[0169] The data determination module is configured to determine a current robot arm end touch pen position and a robot arm end touch pen speed.

[0170] The current robot arm environment state determination module is configured to determine that a robot arm motion range and environment information, position information of the target region, the current robot arm end touch pen position and the robot arm end touch pen speed are the current robot arm environment state.

[0171] The best robot arm end touch pen speed sequence determination module is configured to input the current robot arm environment state into the second reinforcement learning model for processing to obtain a best robot arm end touch pen speed sequence of the robot arm end touch pen moving to the target region.

[0172] a test module 605, configured to control the robot arm to move the robot arm end touch pen along the optimal robot arm end touch pen speed sequence to the target area for automatic testing based on the optimal robot arm end touch pen speed sequence.

[0173] Optionally, the system 600 further includes:

[0174] a first model construction module, configured to construct a first reinforcement learning model;

[0175] a first model training module, configured to input a training sample environment state into the first reinforcement learning model for model training, the training sample environment state being an environment state composed of a car machine interface feature, a sliding box feature and a target area feature;

[0176] a first execution action determination module, configured to calculate a Q value of each action through a Q value prediction network in the first reinforcement learning model according to the training sample environment state, and select an execution action based on an action selection strategy;

[0177] a first data determination module, configured to determine a next moment training sample environment state and an immediate reward of the training sample environment state based on the selected execution action;

[0178] a first experience tuple determination and storage module, configured to determine the training sample environment state, the execution action, the immediate reward and the next moment training sample environment state as an experience tuple, and store the experience tuple into an experience replay buffer pool;

[0179] a first model parameter updating module, configured to randomly sample a batch of experience tuples from the experience replay buffer pool, train the Q value prediction network, update network parameters of the Q value prediction network until performance of the Q value prediction network is stable, and obtain a trained first reinforcement learning model.

[0180] Optionally, the first execution action determination module includes:

[0181] a first Q value determination module, configured to calculate a Q value of each action through a Q value prediction network algorithm in the first reinforcement learning model according to the training sample environment state;

[0182] a first execution action determination sub-module, configured to select an execution action from the each action according to the Q value of each action, an e-greedy strategy and an action masking algorithm.

[0183] Optionally, the first model parameter updating module includes:

[0184] a first optimal action determination module configured to randomly sample a batch of experience tuples from the experience replay buffer, and select, for each training sample environment state recorded in each experience tuple, an optimal action at the training sample environment state in each next time instant by an ε-greedy policy;

[0185] a first target Q value determination module configured to calculate a target Q value for each optimal action by a target network in the first reinforcement learning model;

[0186] a Q value prediction network parameter update module configured to update parameters of the Q value prediction network according to an error between a Q value corresponding to the same experience tuple and the target Q value;

[0187] a first target network parameter update module configured to copy the parameters of the Q value prediction network to the target network to update parameters of the target network after every first time length;

[0188] a first model determination module configured to obtain a trained first reinforcement learning model until performance of the Q value prediction network is stable.

[0189] Optionally, the system 600 further comprises:

[0190] a second model construction module configured to construct a second reinforcement learning model;

[0191] a second model training module configured to input a training sample environment state to the first reinforcement learning model for model training, the training sample environment state being an environment state composed of a mechanical arm movement range and environment information, position information of a target region, a position of a mechanical arm end touch pen, and a speed of the mechanical arm end touch pen;

[0192] a second execution action determination module configured to determine an execution action by a policy network in the second reinforcement learning model according to the training sample environment state;

[0193] a second data determination module configured to determine a next time instant training sample environment state and an immediate reward of the training sample environment state based on the determined execution action;

[0194] a second experience tuple determination and storage module configured to determine the training sample environment state, the execution action, the immediate reward, and the next time instant training sample environment state as an experience tuple, and store the experience tuple to an experience replay buffer;

[0195] The second model parameter updating module is configured to randomly sample a batch of experience tuples from the experience replay buffer, train the policy network and the value network in the second reinforcement learning model, update network parameters of the policy network and the value network until performance of the second reinforcement learning model is stable, and obtain a trained second reinforcement learning model.

[0196] Optionally, the second execution action determining module comprises:

[0197] The initial execution action determining module is configured to determine an initial execution action according to the training sample environment state by using a policy network in the second reinforcement learning model.

[0198] The noise adding module is configured to add random noise to the initial execution action to obtain a final execution action.

[0199] Optionally, the second model parameter updating module comprises:

[0200] The second Q value determining module is configured to randomly sample a batch of experience tuples from the experience replay buffer, and determine a Q value of each experience tuple by using a value network.

[0201] The second execution action determining submodule is configured to determine a target execution action corresponding to each experience tuple according to a training sample environment state at a next time in each experience tuple by using a target policy network in the second reinforcement learning model.

[0202] The second target Q value determining module is configured to calculate a target Q value corresponding to the training sample environment state at the next time and the target execution action in each experience tuple by using a target value network based on the determined target execution action corresponding to each experience tuple.

[0203] The value network parameter updating module is configured to update parameters of the value network based on a difference between the Q value and the target Q value corresponding to the same experience tuple.

[0204] The policy network parameter updating module is configured to update parameters of the policy network based on gradient information of the value network on the action.

[0205] The second target network parameter updating module is configured to copy the parameters of the policy network to a target policy network and copy the parameters of the value network to a target value network to update the parameters of the target policy network and the target value network after every second time length.

[0206] The second model determining module is configured to obtain a trained second reinforcement learning model until performance of the policy network and the value network in the second reinforcement learning model is stable.

[0207] Based on the same inventive concept, one embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the steps in the method for automatically testing a UI according to the first aspect of the present application.

[0208] Based on the same inventive concept, one embodiment of the present application provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the steps in the method for automatically testing a UI according to the first aspect of the present application.

[0209] For the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the description of the method embodiment.

[0210] It should be noted that, for the method embodiment, in order to simply describe, the method embodiment is described as a series of action combinations, but those skilled in the art should know that the method embodiment is not limited to the described action sequence, because according to the method embodiment, some steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions involved are not necessarily required by the method embodiment.

[0211] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same and similar parts between the embodiments can be referred to each other.

[0212] Those skilled in the art should know that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0213] The embodiments of the present application are described with reference to the flowchart illustrations and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It is understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0214] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal devices to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0215] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal devices, such that a series of operational steps are carried out on the computer or other programmable terminal devices to produce a computer implemented process so that the instructions executed on the computer or other programmable terminal devices provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0216] Although the preferred embodiments of the present application have been described, those skilled in the art will be able to make additional modifications and variations to these embodiments once they have the benefit of the foregoing description. Accordingly, it is intended that the appended claims be construed to include all such modifications and variations as fall within the scope of the embodiments of the present application.

[0217] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0218] The above provides a detailed description of the method, system and product of the UI automatic test provided by the present application. The principles and implementation modes of the present application are described by using specific examples. The above description of the examples is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation modes and application ranges will be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for automated UI testing, characterized in that, The method includes: The feature extraction network is used to extract features from the vehicle interface, the starting sliding box, and the target area to be tested, respectively, to obtain the features of the vehicle interface, the sliding box, and the target area. The sliding box and the target area are the same size. The vehicle interface features, the sliding box features, and the target area features are connected to obtain connection features; The connection feature is determined to be the vehicle interface environment state, and is input into the first reinforcement learning model for processing to obtain the position information of the target area in the vehicle interface; the first reinforcement learning model is a reinforcement learning model based on a deep Q network; The current robotic arm environment state is input into the second reinforcement learning model for processing to obtain the optimal motion trajectory of the stylus at the end of the robotic arm to the target area. The current robotic arm environment state includes the position information of the target area in the vehicle interface and the spatial position information of the target area inside the vehicle. The second reinforcement learning model is a reinforcement learning model based on deep deterministic policy gradient. Based on the optimal motion trajectory, the end effector of the robotic arm is controlled to move to the target area for automatic testing.

2. The method for automatic UI testing according to claim 1, characterized in that, Feature extraction networks are used to extract features from the vehicle interface, the initial sliding box, and the target region to be tested, respectively, to obtain features of the vehicle interface, the sliding box, and the target region, including: The vehicle interface, the initial sliding box, and the target area to be tested are input into the feature extraction network for feature extraction. The spatial pyramid pooling layer of the feature extraction network unifies the vehicle interface features, sliding box features, and target region features to the same scale.

3. The method for automatic UI testing according to claim 1, characterized in that, The current state of the robotic arm's environment is input into the second reinforcement learning model for processing to obtain the optimal motion trajectory of the stylus at the end of the robotic arm to the target area, including: Determine the current position and speed of the stylus at the end of the robotic arm; The current robotic arm environmental state is determined by the robotic arm's range of motion and environmental information, the target area's location information, the current position of the robotic arm's end effector, and the speed of the robotic arm's end effector. The current robotic arm environment state is input into the second reinforcement learning model for processing to obtain the optimal robotic arm end-effector stylus speed sequence for moving the robotic arm end-effector stylus to the target area. The automatic testing process, based on the optimal motion trajectory, involves controlling the end effector of the robotic arm to move to the target area, including: Based on the optimal stylus speed sequence at the end of the robotic arm, the stylus at the end of the robotic arm is controlled by the robotic arm to move along the optimal stylus speed sequence to the target area for automatic testing.

4. The method for automatic UI testing according to claim 1, characterized in that, The method further includes: Construct the first reinforcement learning model; The training sample environment state is input into the first reinforcement learning model for model training. The training sample environment state is an environment state composed of vehicle interface features, sliding box features, and target region features. Based on the environmental state of the training samples, the Q-value of each action is calculated by the Q-value prediction network in the first reinforcement learning model, and the action to be executed is selected based on the action selection strategy. Based on the selected action, determine the next moment's training sample environment state and immediate reward. The training sample environment state, the executed action, the immediate reward, and the training sample environment state at the next moment are determined as experience tuples and stored in the experience replay buffer pool. A batch of experience tuples is randomly sampled from the experience replay buffer pool to train the Q-value prediction network. The network parameters of the Q-value prediction network are updated until the performance of the Q-value prediction network stabilizes, thus obtaining a qualified first reinforcement learning model.

5. The method for automatic UI testing according to claim 4, characterized in that, Based on the environmental state of the training samples, the Q-value of each action is calculated through the Q-value prediction network in the first reinforcement learning model, and the action to be executed is selected based on the action selection strategy, including: Based on the environmental state of the training samples, the Q value of each action is calculated using the Q-value prediction network algorithm in the first reinforcement learning model; Based on prior rules, non-optimal actions are determined for each action, and a Q value less than a set threshold is assigned to the determined non-optimal actions; Based on the Q-value of each action and a greedy strategy, select the action to be executed from each action.

6. The method for automatic UI testing according to claim 4, characterized in that, A batch of empirical tuples is randomly sampled from the empirical replay buffer to train the Q-value prediction network. The network parameters of the Q-value prediction network are updated until the performance of the Q-value prediction network stabilizes, thus obtaining a first reinforcement learning model that has passed training. A batch of experience tuples are randomly sampled from the experience replay buffer pool. For the next time step training sample environment state recorded in each experience tuple, the best action is selected in each next time step training sample environment state by a greedy strategy. The best action is the action corresponding to the maximum Q value obtained by calculating the Q value of each action through the target network in the first reinforcement learning model. The parameters of the Q-value prediction network are updated based on the error between the Q-value corresponding to the same empirical tuple and the target Q-value, where the target Q-value is the Q-value of the optimal action. After each first time interval, the parameters of the Q-value prediction network are copied to the target network to update the parameters of the target network; The first reinforcement learning model is obtained when the performance of the Q-value prediction network stabilizes.

7. The method for automatic UI testing according to claim 1, characterized in that, The method further includes: Construct a second reinforcement learning model; The training sample environment state is input into the second reinforcement learning model for model training. The training sample environment state is an environment state composed of the robotic arm's range of motion and environmental information, the target area's position information, the position of the robotic arm's end-point stylus, and the robotic arm's end-point stylus speed. Based on the environmental state of the training samples, the action to be performed is determined through the policy network in the second reinforcement learning model. Based on the determined execution action, determine the next moment's training sample environment state and immediate reward. The training sample environment state, the executed action, the immediate reward, and the training sample environment state at the next moment are determined as experience tuples and stored in the experience replay buffer pool. A batch of experience tuples are randomly sampled from the experience replay buffer pool, and the policy network and value network in the second reinforcement learning model are trained. The network parameters of the policy network and value network are updated until the performance of the second reinforcement learning model is stable, thus obtaining a qualified second reinforcement learning model.

8. The method for automatic UI testing according to claim 7, characterized in that, Based on the environmental state of the training samples, the action to be executed is determined through the policy network in the second reinforcement learning model, including: Based on the environmental state of the training samples, the initial execution action is determined through the policy network in the second reinforcement learning model; Random noise is added to the initial execution action to obtain the final execution action.

9. A method for automatic UI testing according to claim 7, characterized in that, A batch of experience tuples is randomly sampled from the experience replay buffer to train the policy network and value network in the second reinforcement learning model. The network parameters of the policy network and value network are updated until the performance of the second reinforcement learning model stabilizes, thus obtaining a qualified second reinforcement learning model, including: A batch of experience tuples is randomly sampled from the experience replay buffer pool, and the Q value of each experience tuple is determined through a value network. Based on the environment state of the training samples in each experience tuple at the next time step, the target execution action corresponding to each experience tuple is determined by the target policy network in the second reinforcement learning model. Based on the target execution action corresponding to each determined experience tuple, the target Q value corresponding to the training sample environment state and target execution action in the next time step is calculated through the target value network. The parameters of the value network are updated based on the difference between the Q-values ​​corresponding to the same empirical tuples and the target Q-values. The parameters of the policy network are updated based on the gradient information of the action from the value network. After every second time interval, the parameters of the policy network are copied to the target policy network, and the parameters of the value network are copied to the target value network to update the parameters of the target policy network and the target value network. The training continues until the policy network and value network in the second reinforcement learning model stabilize, at which point a qualified second reinforcement learning model is obtained.

10. A system for automatic UI testing, characterized in that, The system includes: The feature extraction module is used to extract features from the vehicle interface, the starting sliding box, and the target area to be tested through a feature extraction network, respectively, to obtain vehicle interface features, sliding box features, and target area features, wherein the sliding box and the target area are the same size; The feature connection module is used to perform feature connection on the vehicle interface features, the sliding box features, and the target area features to obtain connection features; The target region identification and localization module is used to determine that the connection feature is the vehicle interface environment state, input it to the first reinforcement learning model for processing, and obtain the position information of the target region in the vehicle interface; the first reinforcement learning model is a reinforcement learning model based on a deep Q network; The motion trajectory determination module is used to input the current robotic arm environment state into the second reinforcement learning model for processing, and obtain the optimal motion trajectory of the stylus at the end of the robotic arm moving to the target area. The current robotic arm environment state includes the position information of the target area in the vehicle interface and the spatial position information of the target area inside the vehicle. The second reinforcement learning model is a reinforcement learning model based on deep deterministic policy gradient. The testing module is used to control the end effector of the robotic arm to move to the target area for automatic testing based on the optimal motion trajectory.

11. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and running on the processor, wherein the computer program, when executed by the processor, implements the steps of a method for automated UI testing as claimed in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a method for automated UI testing as described in any one of claims 1 to 9.

Images