UI automatic test method, system and product

Abstract

The embodiment of the invention provides a UI automatic test method, system and product, and the method comprises the steps: a feature extraction network carries out the feature extraction of a vehicle-mounted terminal interface, an initial sliding frame and a to-be-tested target region, obtains and connects the features of the vehicle-mounted terminal interface, the features of the sliding frame and the features of the target region, and obtains the connection features; the connection features are input into a first reinforcement learning model for processing, and position information of the target area in the vehicle-machine interface is obtained; the current environment state of the mechanical arm is input into a second reinforcement learning model to be processed, the optimal motion trail of a touch pen at the tail end of the mechanical arm moving to the target area is obtained, and the current environment state of the mechanical arm comprises position information of the target area in the vehicle-machine interface; and based on the optimal motion track, controlling a tail end touch pen of the mechanical arm to move to the target area for automatic testing. The objective of the invention is to reduce the time complexity of target area identification and positioning and improve the identification effect, thereby improving the efficiency and effect of automatic testing.

Description

Technical Field

[0001] The present application relates to the field of UI testing technology, and in particular to a method, system and product for automatic UI testing. Background Art

[0002] Ensuring the stability of the user-onboard computer interaction interface is very important for improving software quality and user satisfaction. The existing UI automation test of the car computer interface is an automated testing technology based on the user interface. It simulates user behaviors, such as clicks, based on the identification and positioning of interface elements to verify whether the software interface meets expectations. It plays an important role in improving test efficiency, reducing costs, and ensuring test accuracy and consistency. Existing element identification and positioning methods include keyword-driven methods, UI control search-based methods, and image matching-based methods.

[0003] The image matching method can identify and locate the target control by finding the image area in the current interface that is the same or similar to the given target area to be controlled. Because it is not easily disturbed by code logic, its usage rate is gradually increasing. However, most of the existing image matching methods are based on traditional algorithms based on template matching and feature point matching. Such traditional image target matching methods traverse the entire image until the target area is found, which has high time complexity and poor recognition effect. Summary of the invention

[0004] In view of this, the embodiments of the present application provide a method, system and product for UI automatic testing, aiming to reduce the time complexity of target area identification and positioning and improve the identification effect, thereby improving the efficiency and effect of automated testing.

[0005] A first aspect of an embodiment of the present application provides a method for automatic UI testing, the method comprising:

[0006] The vehicle interface, the starting sliding frame and the target area to be measured are respectively extracted by a feature extraction network to obtain vehicle interface features, sliding frame features and target area features, wherein the sliding frame has the same size as the target area;

[0007] Performing feature connection on the vehicle machine interface feature, the sliding frame feature and the target area feature to obtain a connection feature;

[0008] Determine the connection feature as a vehicle-machine interface environment state, input it into a first reinforcement learning model for processing, and obtain position information of the target area in the vehicle-machine interface;

[0009] Inputting the current robot arm environment state into the second reinforcement learning model for processing to obtain the optimal motion trajectory of the stylus at the end of the robot arm to the target area, wherein the current robot arm environment state includes the position information of the target area in the vehicle interface and the spatial position information of the target area in the vehicle;

[0010] Based on the optimal motion trajectory, the end stylus of the robotic arm is controlled to move to the target area for automatic testing.

[0011] Optionally, feature extraction is performed on the vehicle interface, the starting sliding frame, and the target area to be measured respectively through a feature extraction network to obtain vehicle interface features, sliding frame features, and target area features, including:

[0012] The vehicle computer interface, the starting sliding frame and the target area to be measured are input into the feature extraction network for feature extraction;

[0013] The vehicle interface features, sliding box features and target area features are unified to the same scale through the spatial pyramid pooling layer of the feature extraction network.

[0014] Optionally, the current robot arm environment state is input into a second reinforcement learning model for processing to obtain an optimal motion trajectory of the stylus at the end of the robot arm moving to the target area, including:

[0015] Determine the current end-of-arm stylus position and end-of-arm stylus speed;

[0016] Determine the robot arm motion range and environmental information, the location information of the target area, the current robot arm end stylus position and the robot arm end stylus speed as the current robot arm environmental state;

[0017] Inputting the current robot arm environment state into a second reinforcement learning model for processing to obtain an optimal robot arm end stylus velocity sequence for moving the robot arm end stylus to the target area;

[0018] The step of controlling the end stylus of the robotic arm to move to the target area for automatic testing based on the optimal motion trajectory includes:

[0019] Based on the optimal robot arm end stylus speed sequence, the robot arm controls the robot arm end stylus to move along the optimal robot arm end stylus speed sequence to the target area for automatic testing.

[0020] Optionally, the method further includes:

[0021] Build the first reinforcement learning model;

[0022] Inputting a training sample environment state into the first reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of vehicle-machine interface features, sliding box features, and target area features;

[0023] According to the training sample environment state, the Q value of each action is calculated by the Q value prediction network in the first reinforcement learning model, and the execution action is selected based on the action selection strategy;

[0024] Based on the selected execution action, determining the training sample environment state and the immediate reward at the next moment of the training sample environment state;

[0025] Determine the training sample environment state, the execution action, the immediate reward and the training sample environment state at the next moment as an experience tuple, and store it in an experience playback buffer pool;

[0026] A batch of experience tuples are randomly sampled from the experience replay buffer pool, the Q-value prediction network is trained, and the network parameters of the Q-value prediction network are updated until the performance of the Q-value prediction network is stable, thereby obtaining a first reinforcement learning model that has passed the training.

[0027] Optionally, according to the training sample environment state, the Q value of each action is calculated by a Q value prediction network in the first reinforcement learning model, and an execution action is selected based on an action selection strategy, including:

[0028] According to the training sample environment state, the Q value of each action is calculated by the Q value prediction network algorithm in the first reinforcement learning model;

[0029] Determine a non-preferred action in each of the actions based on a priori rules, and assign a Q value less than a set threshold to the determined non-preferred action;

[0030] According to the Q value of each action, the ε-greedy strategy selects an action to be executed from each action.

[0031] Optionally, randomly sampling a batch of experience tuples from the experience replay buffer pool, training the Q-value prediction network, and updating the network parameters of the Q-value prediction network until the performance of the Q-value prediction network is stable, thereby obtaining a first reinforcement learning model that has passed the training, including:

[0032] A batch of experience tuples are randomly sampled from the experience replay buffer pool, and for the training sample environment state at the next moment recorded in each experience tuple, the best action under the training sample environment state at each next moment is selected through an ε-greedy strategy, wherein the best action is the action corresponding to the maximum Q value obtained by calculating the Q value of each action through the target network in the first reinforcement learning model;

[0033] updating the parameters of the Q-value prediction network according to the error between the Q-value corresponding to the same experience tuple and the target Q-value, wherein the target Q-value is the Q-value of the optimal action;

[0034] After every first time period, copying the parameters of the Q-value prediction network to the target network to update the parameters of the target network;

[0035] Until the performance of the Q-value prediction network is stabilized, a first reinforcement learning model that has passed the training is obtained.

[0036] Optionally, the method further includes:

[0037] Building a second reinforcement learning model;

[0038] Inputting the training sample environment state into the second reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of the robot arm motion range and environment information, the position information of the target area, the robot arm end stylus position and the robot arm end stylus speed;

[0039] Determining an execution action through a policy network in a second reinforcement learning model according to the training sample environment state;

[0040] Based on the determined execution action, determining the training sample environment state and the immediate reward at the next moment of the training sample environment state;

[0041] Determine the training sample environment state, the execution action, the immediate reward and the training sample environment state at the next moment as an experience tuple, and store it in an experience playback buffer pool;

[0042] A batch of experience tuples are randomly sampled from the experience replay buffer pool, the policy network and the value network in the second reinforcement learning model are trained, and the network parameters of the policy network and the value network are updated until the performance of the second reinforcement learning model is stable, thereby obtaining a qualified second reinforcement learning model.

[0043] Optionally, determining the execution action through a policy network in a second reinforcement learning model according to the training sample environment state includes:

[0044] Determining an initial execution action through a policy network in a second reinforcement learning model according to the training sample environment state;

[0045] Random noise is added to the initial execution action to obtain a final execution action.

[0046] Optionally, randomly sampling a batch of experience tuples from the experience replay buffer pool, training the policy network and the value network in the second reinforcement learning model, and updating the network parameters of the policy network and the value network until the performance of the second reinforcement learning model is stable, to obtain a second reinforcement learning model that has been trained to be qualified, including:

[0047] Randomly sampling a batch of experience tuples from the experience replay buffer pool, and determining the Q value of each experience tuple through the value network;

[0048] According to the training sample environment state at the next moment in each experience tuple, the target execution action corresponding to each experience tuple is determined through the target strategy network in the second reinforcement learning model;

[0049] Based on the target execution action corresponding to each determined experience tuple, the target Q value corresponding to the training sample environment state and the target execution action at the next moment in each experience tuple is calculated through the target value network;

[0050] Update the parameters of the value network based on the difference between the Q value corresponding to the same experience tuple and the target Q value;

[0051] Update the parameters of the policy network based on the gradient information of the value network to the action;

[0052] After every second time period, copy the parameters of the policy network to the target policy network, and copy the parameters of the value network to the target value network, so as to update the parameters of the target policy network and the target value network;

[0053] Until the performance of the policy network and the value network in the second reinforcement learning model is stable, a qualified second reinforcement learning model is obtained.

[0054] A second aspect of an embodiment of the present application provides a system for automatic UI testing, the system comprising:

[0055] A feature extraction module is used to extract features of the vehicle interface, the starting sliding frame and the target area to be measured respectively through a feature extraction network to obtain vehicle interface features, sliding frame features and target area features, wherein the sliding frame has the same size as the target area;

[0056] A feature connection module, used for performing feature connection on the vehicle interface feature, the sliding frame feature and the target area feature to obtain a connection feature;

[0057] A target area recognition and positioning module, used to determine that the connection feature is a vehicle-machine interface environment state, input it into a first reinforcement learning model for processing, and obtain the location information of the target area in the vehicle-machine interface;

[0058] a motion trajectory determination module, used to input the current robot arm environment state into the second reinforcement learning model for processing, and obtain the optimal motion trajectory of the stylus at the end of the robot arm moving to the target area, wherein the current robot arm environment state includes the position information of the target area in the vehicle interface and the spatial position information of the target area in the vehicle;

[0059] The testing module is used to control the end stylus of the robot arm to move to the target area for automatic testing based on the optimal motion trajectory.

[0060] A third aspect of an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and running on the processor, wherein when the computer program is executed by the processor, the steps in a method for automatic UI testing as described in the first aspect of the present application are implemented.

[0061] The fourth aspect of an embodiment of the present application provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the method for automatic UI testing as described in the first aspect of the present application are implemented.

[0062] The UI automatic testing method provided by this application has the following advantages:

[0063] A method for automatic UI testing provided by an embodiment of the present application comprises the following steps: first, feature extraction is performed on a vehicle-machine interface, a starting sliding box, and a target area to be tested respectively through a feature extraction network to obtain vehicle-machine interface features, sliding box features, and target area features, wherein the sliding box has the same size as the target area; feature connection is performed on the vehicle-machine interface features, the sliding box features, and the target area features to obtain connection features; the connection features are determined to be the vehicle-machine interface environment state, and are input into a first reinforcement learning model for processing to obtain position information of the target area in the vehicle-machine interface; the current robotic arm environment state is input into a second reinforcement learning model for processing to obtain an optimal motion trajectory of the robotic arm end stylus moving to the target area, wherein the current robotic arm environment state includes position information of the target area in the vehicle-machine interface and spatial position information of the target area in the vehicle; based on the optimal motion trajectory, the robotic arm end stylus is controlled to move to the target area for automatic testing. Therefore, this application introduces a reinforcement learning model, and determines the target area to be operated by inputting the current vehicle interface environment state into the reinforcement learning model without traversing the entire image, thereby reducing time complexity. At the same time, as the quality of the sample data of the training model is improved, the recognition effect of this method can also be greatly improved. At the same time, the reinforcement learning model is also used to determine the optimal motion trajectory of the stylus at the end of the robotic arm from the current position to the position of the target area, thereby improving the test efficiency of moving the stylus at the end of the robotic arm to the target position for automated testing. At the same time, as the recognition effect of the position recognition and positioning of the target area is improved, the test effect of the automated test is also improved accordingly. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor.

[0065] Figure 1 A flowchart of a method for automatic UI testing is shown in one embodiment of the present application;

[0066] Figure 2 A schematic diagram of a vehicle computer interface in a method for automatic UI testing according to an embodiment of the present application;

[0067] Figure 3 A schematic diagram of sliding a sliding box in a method for automatic UI testing according to an embodiment of the present application;

[0068] Figure 4A structural diagram of a first reinforcement learning model in a method for automatic UI testing shown in one embodiment of the present application;

[0069] Figure 5 Another flow chart of a method for automatic UI testing shown in one embodiment of the present application;

[0070] Figure 6 A schematic diagram of a UI automatic testing system is shown as an embodiment of the present application. DETAILED DESCRIPTION

[0071] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0072] refer to Figure 1 , Figure 1 This is a flow chart of a method for automatic UI testing according to an embodiment of the present application. Figure 1 As shown, the method includes:

[0073] Step S1: extract features of the vehicle interface, the starting sliding frame and the target area to be measured respectively through a feature extraction network to obtain vehicle interface features, sliding frame features and target area features, wherein the sliding frame has the same size as the target area.

[0074] In this embodiment, when the UI interface of the vehicle is automatically tested, there are corresponding test cases according to different test contents. The test cases record the image data of the target area of ​​the UI interface to be tested. Each test case starts with the same initial interface. Each time a new interface is entered, the search starts from the specified starting point L0. The starting time of the sliding box is the image data of the same size as the image of the target area generated at the specified starting point L0, such as Figure 2 As shown. Based on the test case, the image data of the target area corresponding to the test case is determined. Then, through the feature extraction network in the first reinforcement learning model used to determine the specific position of the target area in the vehicle-machine interface, the image data of the target area, the image data of the vehicle-machine interface in the initial state, and the image data of the sliding box in the initial state are respectively extracted to obtain the target area features corresponding to the target area, the vehicle-machine interface features corresponding to the vehicle-machine interface, and the sliding box features corresponding to the sliding box.

[0075] Step S2: feature-connecting the vehicle-machine interface feature, the sliding frame feature and the target area feature to obtain a connection feature.

[0076] In this embodiment, the vehicle interface feature, the sliding frame feature and the target area feature obtained in step S1 are feature-connected to obtain a connected feature.

[0077] Step S3: Determine that the connection feature is the vehicle-machine interface environment state, input it into the first reinforcement learning model for processing, and obtain the position information of the target area in the vehicle-machine interface.

[0078] In this embodiment, after the corresponding connection feature is obtained through step S2, the connection feature is used as the current vehicle interface environment state. In this vehicle interface environment state, the vehicle interface belongs to the environment, the sliding box belongs to the intelligent agent that performs the action, and the target area feature belongs to the position that the intelligent agent (that is, the sliding box) needs to reach by performing the corresponding action, that is, the target. The actions performed by the intelligent agent include up, down, left, and right. The range of each movement is preferably 1% of the image data of the vehicle interface. Figure 3 As shown, the vehicle-machine interface environment state is then input into the first reinforcement learning model for processing. The first reinforcement learning model will obtain the action sequence A of the intelligent agent (that is, the sliding box) in the environment (that is, the vehicle-machine interface) to move to the target area based on the vehicle-machine interface environment state. Based on the action sequence A to move to the target area and the position information of the designated starting point L0 where the sliding box is located in the starting state, the position information of the target area in the vehicle-machine interface is determined to be L0+A. Among them, the first reinforcement learning model is preferably a DQN (Deep Q-Network) reinforcement learning algorithm. It should be understood that the first reinforcement learning model can also be other reinforcement learning algorithms, which are not specifically limited here.

[0079] Step S4: Input the current robot arm environment state into the second reinforcement learning model for processing to obtain the optimal motion trajectory of the stylus at the end of the robot arm to the target area, wherein the current robot arm environment state includes the position information of the target area in the vehicle interface and the spatial position information of the target area in the vehicle.

[0080] In this embodiment, after obtaining the position information of the target area in the vehicle-computer interface through step S3, the current robot arm environment state is determined, and the environment state at least includes the position information of the target area in the vehicle-computer interface to which the stylus at the end of the robot arm needs to move and the spatial position information of the target area in the vehicle. The determined current robot arm environment state is then input into the second reinforcement learning model for processing, and the second reinforcement learning model will determine the optimal motion trajectory for the stylus at the end of the robot arm to move from its current position to the position of the target area based on the current robot arm environment state. Among them, the second reinforcement learning model is preferably a deep deterministic policy gradient algorithm (Deep Deterministic Policy Gradient, DDPG) based on reinforcement learning. It should be understood that the second reinforcement learning model can also be other reinforcement learning algorithms, which are not specifically limited here.

[0081] Step S5: Based on the optimal motion trajectory, control the stylus at the end of the robotic arm to move to the target area for automatic testing.

[0082] In this embodiment, after obtaining the optimal motion trajectory of the stylus at the end of the robotic arm moving from its current position to the location of the target area through step S4, based on the optimal motion trajectory, the stylus at the end of the robotic arm is controlled to move along the motion trajectory to the location of the target area to perform automatic testing on the vehicle interface, such as click testing, sliding testing, etc.

[0083] The embodiment of the present application provides a method for automatic testing of UI. First, the feature extraction network is used to extract features of the vehicle-machine interface, the starting sliding box and the target area to be tested respectively, so as to obtain the vehicle-machine interface features, the sliding box features and the target area features, and the sliding box has the same size as the target area; the vehicle-machine interface features, the sliding box features and the target area features are feature-connected to obtain connection features; the connection features are determined to be the vehicle-machine interface environment state, and are input into the first reinforcement learning model for processing to obtain the position information of the target area in the vehicle-machine interface; the current robot arm environment state is input into the second reinforcement learning model for processing to obtain the motion trajectory of the stylus at the end of the robot arm moving to the target area; based on the motion trajectory, the robot arm is controlled to move to the target area for automatic testing. Therefore, the present application introduces a reinforcement learning model, and determines the target area to be operated by inputting the current vehicle-machine interface environment state into the reinforcement learning model without traversing the entire image, thereby reducing the time complexity. At the same time, along with the improvement of the quality of the sample data of the training model, the recognition effect of this method can also be greatly improved. At the same time, the reinforcement learning model is also used to determine the optimal motion trajectory of the stylus at the end of the robot arm from the current position to the location of the target area, thereby improving the test efficiency of moving the stylus at the end of the robot arm to the target position for automated testing. At the same time, as the recognition effect of the position recognition and positioning of the target area is improved, the test effect of the automated test is also improved accordingly.

[0084] In combination with the above embodiments, in one implementation, the embodiment of the present application further provides a method for automatic UI testing. In the method for automatic UI testing, step S1 may include: inputting the vehicle interface, the starting sliding box, and the target area to be tested into a feature extraction network for feature extraction; and unifying the vehicle interface features, the sliding box features, and the target area features to the same scale through the spatial pyramid pooling layer of the feature extraction network.

[0085] In this embodiment, since the size of the image data of the vehicle interface is inconsistent with the image data size of the target area and the sliding frame, the present application adjusts the feature extraction network in the first reinforcement learning model to facilitate the subsequent full-connection layer network input, so that the adjusted feature extraction network can output the same scale results for input images of different scales. Specifically: a spatial pyramid pooling layer (Spatial Pyramid Pooling, SPP) is set after the last pooling layer of the feature extraction network in the first reinforcement learning model to unify the scale of the output features. Among them, the preferred network structure of the feature extraction network is five convolutional layers + Relu layer + pooling layer + one SPP layer, such as Figure 4As shown in the figure, the image data of the vehicle interface, the image data of the starting sliding frame, and the image data of the target area to be measured are respectively input into the feature extraction network for feature extraction. All layers before the SPP layer perform feature extraction to obtain the corresponding features, and then the last SPP layer unifies the feature scale to obtain the vehicle interface features, sliding frame features, and target area features of the same scale.

[0086] In combination with the above embodiments, in one implementation, the present application embodiment further provides a method for automatic UI testing. In the method for automatic UI testing, step S4 may include steps S41 to S43:

[0087] Step S41: Determine the current position of the stylus at the end of the robot arm and the speed of the stylus at the end of the robot arm.

[0088] In this embodiment, the current robot arm environment state input to the second reinforcement learning model for processing includes information such as the current robot arm end stylus position, the current robot arm end stylus speed, the robot arm motion range and environment information and the location information of the target area. The robot arm can be fixed in the car to perform UI interface testing in a fixed operating space. A silicone stylus is fixed at the end of the robot arm to prevent excessive force from damaging the screen. The current position of the stylus at the end of the robotic arm refers to the current spatial position of the tip of the silicone stylus at the end of the robotic arm; the current speed of the stylus at the end of the robotic arm refers to the three-dimensional spatial vector of the tip of the silicone stylus at the end of the robotic arm moving from the spatial position at the previous moment to the spatial position at the current moment. For example, if the spatial position of the stylus at the end of the robotic arm at the previous moment is (a1, b1, c1), and the spatial position of the stylus at the end of the robotic arm at the current moment is (a2, b2, c2), then the current speed of the stylus at the end of the robotic arm refers to the spatial vector pointing from the spatial position (a1, b1, c1) to the spatial position (a2, b2, c2). The spatial vector includes a movement length and a movement direction, and the movement length is the length between the two spatial positions. , the movement direction is from the spatial position (a1, b1, c1) to the spatial position (a2, b2, c2); the movement range and environmental information of the robotic arm refers to the information of the spatial range in which the robotic arm can move after being fixed in the car. The spatial range can be a spatial sphere, a spatial cuboid, or a spatial cube, etc., which is not specifically limited here; the position information of the target area is the position information of the area where the UI interface test will be performed determined by the first reinforcement learning model. Although this information belongs to the position information of the target area relative to the vehicle interface, the robotic arm is fixed in the car. Based on the relative position relationship between the robotic arm and the vehicle interface and the position information of the target area, the robotic arm can locate the position information of the target area relative to itself.

[0089] In this embodiment, the current stylus position and stylus speed of the end of the robotic arm are determined based on the current state of the robotic arm and the state at the previous moment. Since the robotic arm is fixed in the vehicle, its motion range information is also fixed, so it can be directly obtained.

[0090] Step S42: determining the robot arm motion range and environmental information, the position information of the target area, the current robot arm end stylus position and the robot arm end stylus speed as the current robot arm environmental state.

[0091] In this embodiment, the robot arm motion range and environmental information, the position information of the target area, the current robot arm end stylus position and the current robot arm end stylus speed are determined as the current robot arm environment state. In the robot arm environment state, the robot arm belongs to the intelligent agent, the robot arm end stylus speed belongs to the action of the intelligent agent, the robot arm motion range and environmental information belong to the environment, and the position information of the target area belongs to the position that the intelligent agent needs to reach by executing the action, that is, the target. The movement of each motion node of the robot arm drives the stylus at the end of the robot arm to move in three-dimensional space.

[0092] Step S43: inputting the current robot arm environment state into the second reinforcement learning model for processing, and obtaining an optimal robot arm end stylus speed sequence for the robot arm end stylus to move to the target area.

[0093] In this embodiment, the current robot arm environment state is input into the second reinforcement learning model. The second reinforcement learning model will obtain the best series of actions (that is, the robot arm's end stylus speed sequence) for the intelligent agent (that is, the robot arm) to reach the target area in the environment (that is, the robot arm's motion range and environmental information) based on the current robot arm environment state. The best series of actions is the best robot arm end stylus speed sequence for moving the robot arm end stylus from the current state to the target area.

[0094] In the present application, when step S4 includes steps S41 to S43, step S5 may include: based on the optimal robot arm end stylus speed sequence, controlling the robot arm end stylus along the optimal robot arm end stylus speed sequence to move to the target area for automatic testing by the robot arm.

[0095] In this embodiment, after the optimal robot arm end stylus speed sequence is obtained through the second reinforcement learning model, based on the optimal robot arm end stylus speed sequence, the robot arm controls the robot arm end stylus to move along the optimal robot arm end stylus speed sequence to the target area for automatic testing.

[0096] In combination with the above embodiments, in one implementation, the present application embodiment further provides a method for automatic UI testing. In the method for automatic UI testing, the method further includes steps S01 to S06:

[0097] Step S01: construct a first reinforcement learning model.

[0098] In this embodiment, in the actual application process, the first reinforcement learning model used to determine the location information of the target area in the vehicle interface is a qualified reinforcement learning model. Before its specific application, this application will first train to obtain a qualified first reinforcement learning model. Specifically: construct an initial untrained first reinforcement learning model, which is a DQN (Deep Q-Network) reinforcement learning algorithm.

[0099] Step S02: inputting the training sample environment state into the first reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of vehicle-machine interface features, sliding box features and target area features.

[0100] In this embodiment, the training sample environment state is input into the constructed first reinforcement learning model for model training. The input training sample environment state is also an environment state including vehicle-machine interface features, sliding box features and target area features. At the same time, since the amount of sample data used in the training model is very large, there are multiple types of vehicle-machine interface image data in the image data of the vehicle-machine interface corresponding to the vehicle-machine interface features in a large number of training sample environment states, there are multiple types of sliding box image data in the image data of the sliding box corresponding to the sliding box features, and there are multiple types of target area image data in the image data of the target area corresponding to the target area features, so that the samples of the training model are diverse.

[0101] Step S03: According to the training sample environment state, the Q value of each action is calculated through the Q value prediction network in the first reinforcement learning model, and an execution action is selected based on the action selection strategy.

[0102] In this embodiment, for the training sample environment state input into the constructed first reinforcement learning model, the next action to be performed under the current training sample environment state is selected, and the selection method of this step action is: the Q value prediction network in the first reinforcement learning model is used to calculate the Q value of the up, down, left, and right actions respectively performed in this step, and then based on the action selection strategy related to the action Q value, the specific action to be performed in this step is selected. If this step action has four actions, up, down, left, and right, finally based on the Q values ​​corresponding to the four actions and the action selection strategy, it is finally determined to perform the downward action in this step. Among them, the moving range of each step of the sliding box is preferably one percent of the image data of the vehicle interface.

[0103] Step S04: Based on the selected execution action, determine the training sample environment state and the immediate reward at the next moment of the training sample environment state.

[0104] In this embodiment, based on the execution action selected in step S03, the training sample environment state and the instant reward obtained after the execution action selected in step S03 are executed are determined. The training sample environment state also includes the vehicle interface feature, the sliding box feature and the target area feature. Since the vehicle interface and the target area will not change, after the execution action selected in step S03 is executed, the vehicle interface feature and the target area feature in the training sample environment state will not change, and only the sliding box feature will change. At the same time, after the execution action selected in step S03 is executed, the corresponding instant reward is calculated based on the similarity between the sliding box feature and the target area feature after the execution action selected in step S03 is executed. The higher the similarity between the two, the higher the reward, and the lower the similarity, the lower the reward. At the same time, after the execution action selected in step S03 is executed, the training sample environment state and the instant reward obtained are the training sample environment state and instant reward at the next moment under the training sample environment state currently processed by the constructed first reinforcement learning model.

[0105] Step S05: Determine the training sample environment state, the execution action, the immediate reward and the training sample environment state at the next moment as an experience tuple, and store it in an experience playback buffer pool.

[0106] In this embodiment, after obtaining the training sample environment state and immediate reward at the next moment under the training sample environment state currently processed by the constructed first reinforcement learning model through step S04, the training sample environment state currently processed by the constructed first reinforcement learning model, the execution action determined by step S03, the training sample environment state at the next moment and the immediate reward are determined as an experience tuple and stored in the experience replay buffer pool.

[0107] In this embodiment, since the sliding box slides from the starting position, many steps of action will be performed before sliding to the target area. Therefore, after the constructed first reinforcement learning model determines the execution action and the training sample environment state and immediate reward at the next moment based on the current training sample environment state, the training sample environment state at the next moment is determined as the current training sample environment state, and then the constructed first reinforcement learning model determines the execution action and the training sample environment state and immediate reward at the next moment based on the new current training sample environment state. In this way, a large number of experience tuples can be obtained for a training sample environment state input to the constructed first reinforcement learning model. At the same time, a large number of different training sample environments are input to the constructed first reinforcement learning model, and a larger number of experience tuples will be obtained, so that the experience playback buffer pool will include a large number of experience tuples.

[0108] Step S06: randomly sampling a batch of experience tuples from the experience replay buffer pool, training the Q-value prediction network, and updating the network parameters of the Q-value prediction network until the performance of the Q-value prediction network is stable, thereby obtaining a first reinforcement learning model that has passed the training.

[0109] In this embodiment, by repeatedly executing the above steps S01 to S05, an experience replay buffer pool including a large number of experience tuples is obtained. At this time, a batch of experience tuples are randomly sampled from the experience replay buffer pool to train the Q-value prediction network in the constructed first reinforcement learning model to update the network parameters of the Q-value prediction network. After updating the network parameters of the Q-value prediction network, it is determined whether the performance of the Q-value prediction network is stable. In the case that the performance of the Q-value prediction network is not stable, a batch of experience tuples are continuously randomly sampled from the experience replay buffer pool to train the Q-value prediction network in the constructed first reinforcement learning model to update the network parameters of the Q-value prediction network. Until the performance of the Q-value prediction network is stable, a first reinforcement learning model that has been trained and qualified is obtained.

[0110] In combination with the above embodiments, in one implementation, the present application embodiment further provides a method for automatic UI testing. In the method for automatic UI testing, step S03 may include steps S031 to S032:

[0111] Step S031: According to the training sample environment state, the Q value of each action is calculated by the Q value prediction network algorithm in the first reinforcement learning model.

[0112] In this embodiment, for the training sample environment state input into the constructed first reinforcement learning model, the next action to be performed under the current training sample environment state is selected. At this time, the Q value prediction network algorithm in the constructed first reinforcement learning model is first used to calculate the Q value of each of the four actions of up, down, left and right in the next action based on the current training sample environment state.

[0113] Step S032: Determine the non-preferred action in each action based on the prior rule, and assign a Q value less than a set threshold to the determined non-preferred action; select an execution action from each action according to the Q value of each action and the ε-greedy strategy.

[0114] In this embodiment, after the Q values ​​of the four actions of up, down, left, and right in the next step are calculated based on step S031, the actions that are not desired to be taken are shielded based on the prior rules. Specifically, a large penalty is imposed on the actions that will cause the agent (i.e., the sliding box) to exceed the vehicle interface area, so that the agent avoids selecting the action. These actions that will cause the agent (i.e., the sliding box) to exceed the vehicle interface area are non-preferred actions. For non-preferred actions, the Q value calculated previously is discarded, and a Q value less than the set threshold is assigned to it. The Q value less than the set threshold is a particularly small Q value, so that when the ε-greedy strategy is used to select the action to execute later, this non-preferred action will not be selected. Then, based on the Q values ​​of each action obtained, the execution action is selected from each action through the ε-greedy strategy.

[0115] In combination with the above embodiments, in one implementation, the present application embodiment further provides a method for automatic UI testing. In the method for automatic UI testing, step S06 may include steps S061 to S064:

[0116] Step S061: randomly sample a batch of experience tuples from the experience replay buffer pool, and select the best action under the training sample environment state at each next moment recorded in each experience tuple through an ε-greedy strategy, wherein the best action is the action corresponding to the maximum Q value obtained by calculating the Q value of each action through the target network in the first reinforcement learning model.

[0117] In this embodiment, an experience replay buffer pool including a large number of experience tuples is obtained by repeatedly executing the above steps S01 to S05. At this time, a batch of experience tuples are randomly sampled from the experience replay buffer pool, and for each experience tuple in the batch of experience tuples, the training sample environment state at the next moment after the corresponding action is executed is recorded. At this time, for each training sample environment state at the next moment, the Q value of each action under the training sample environment state at the next moment is calculated through the target network in the first reinforcement learning model, and then the action with the largest Q value under the training sample environment state at the next moment is selected as the best action under the training sample environment state at the next moment through the ε-greedy strategy.

[0118] Step S062: updating the parameters of the Q-value prediction network according to the error between the Q-value corresponding to the same experience tuple and the target Q-value.

[0119] In this embodiment, after obtaining the target Q value under each optimal action through step S061, the network parameters of the Q value prediction network are updated based on the error between the Q value belonging to the same experience tuple (the Q value is the Q value corresponding to the execution action recorded in the same experience tuple) and the target Q value (the target Q value is the Q value corresponding to the optimal action under the training sample environment state at the next moment based on the same experience tuple).

[0120] Step S063: after every first time period, copy the parameters of the Q-value prediction network to the target network to update the parameters of the target network.

[0121] In this embodiment, in order to improve the stability and effect of training, in the process of continuously updating the network parameters of the Q-value prediction network, the latest network parameters of the Q-value prediction network are given to the target network every first time period to update the network parameters of the target network. The first time period can be set according to the actual application scenario and is not specifically limited here.

[0122] Step S064: until the performance of the Q-value prediction network is stabilized, a first reinforcement learning model that has passed the training is obtained.

[0123] In this embodiment, after a batch of randomly sampled experience tuples are used to train the Q-value prediction network, a batch of randomly sampled experience tuples are used to train the Q-value prediction network again until the performance of the trained Q-value prediction network tends to be stable or until the first preset number of trainings are completed, and a first reinforcement learning model that has been trained and qualified is determined to be obtained. At this time, the first reinforcement learning model that has been trained and qualified can be used to determine the location information of the target area in the vehicle-machine interface based on the current vehicle-machine interface environment state. The first preset number of times can be set according to the actual application scenario and is not specifically limited here.

[0124] In combination with the above embodiments, in one implementation, the present application embodiment further provides a method for automatic UI testing. In the method for automatic UI testing, the method further includes steps S001 to S006:

[0125] Step S001: Construct a second reinforcement learning model.

[0126] In this embodiment, in the actual application process, the second reinforcement learning model used to determine the optimal robot arm end stylus speed sequence of the robot arm end stylus is a qualified reinforcement learning model. Before its specific application, this application will first train to obtain a qualified second reinforcement learning model. Specifically: construct an initial untrained second reinforcement learning model, which is a deep deterministic policy gradient algorithm (Deep Deterministic Policy Gradient, DDPG) based on reinforcement learning.

[0127] Step S002: Input the training sample environment state into the second reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of the robot arm motion range and environment information, the position information of the target area, the robot arm end stylus position and the robot arm end stylus speed.

[0128] In this embodiment, the training sample environment state is input into the constructed second reinforcement learning model for model training. The input training sample environment state also includes the robot arm motion range and environment information, the position information of the target area, the position of the stylus at the end of the robot arm and the speed of the stylus at the end of the robot arm. At the same time, since the amount of sample data used in the training model is very large, there are many different position information of the target area in a large number of training sample environment states, there are many different positions of the stylus at the end of the robot arm, and there are many different speeds of the stylus at the end of the robot arm, so that the samples of the training model are diverse.

[0129] Step S003: According to the training sample environment state, determine the execution action through the policy network in the second reinforcement learning model.

[0130] In this embodiment, for the training sample environment state input into the constructed second reinforcement learning model, the next action to be performed under the current training sample environment state is selected, and the step action is selected by determining the execution action through the policy network in the second reinforcement learning model.

[0131] Step S004: Based on the determined execution action, determine the training sample environment state and the immediate reward at the next moment of the training sample environment state.

[0132] In this embodiment, based on the execution action determined in step S003, the training sample environment state and the instant reward obtained after the execution action selected in step S003 is executed are determined. The training sample environment state also includes the robot arm motion range and environment information, the location information of the target area, the location of the stylus at the end of the robot arm, and the speed of the stylus at the end of the robot arm. Since the robot arm motion range and environment information and the location information of the target area will not change, after the execution action selected in step S003 is executed, the robot arm motion range and environment information and the location information of the target area in the training sample environment state will not change, and only the location of the stylus at the end of the robot arm and the speed of the stylus at the end of the robot arm will change. At the same time, after the execution action selected in step S003 is executed, the corresponding instant reward is calculated as follows: when the stylus at the end of the robot arm moves to the location of the target area, the reward r t =10, otherwise reward r t =-|Stylus position at the end of the robotic arm-location of the target area| 2 , if the stylus at the end of the robot arm exceeds the range of motion of the robot arm, an additional penalty r0=-5 is added. At the same time, after executing the execution action selected in step S003, the training sample environment state and the immediate reward obtained are the training sample environment state and immediate reward of the next moment under the training sample environment state currently processed by the constructed second reinforcement learning model.

[0133] Step S005: Determine the training sample environment state, the execution action, the immediate reward and the training sample environment state at the next moment as an experience tuple, and store it in an experience playback buffer pool.

[0134] In this embodiment, after obtaining the training sample environment state and immediate reward at the next moment under the training sample environment state currently processed by the constructed second reinforcement learning model through step S004, the training sample environment state currently processed by the constructed second reinforcement learning model, the execution action determined by step S003, the training sample environment state at the next moment and the immediate reward are determined as an experience tuple and stored in the experience replay buffer pool.

[0135] In this embodiment, since the stylus at the end of the robotic arm moves in three-dimensional space from the starting position, it will perform many steps of action before moving to the location of the target area. Therefore, after the second reinforcement learning model is constructed based on the current training sample environment state, the training sample environment state at the next moment and the immediate reward are determined, and then the training sample environment state at the next moment is determined as the current training sample environment state, and then the second reinforcement learning model is constructed based on the new current training sample environment state. Determine the execution of the action and the training sample environment state at the next moment and the immediate reward. In this way, a large number of experience tuples can be obtained for a training sample environment state input to the constructed second reinforcement learning model. At the same time, a large number of different training sample environments are input to the constructed second reinforcement learning model, and a larger number of experience tuples will be obtained, so that the experience playback buffer pool will include a large number of experience tuples.

[0136] Step S006: Randomly sample a batch of experience tuples from the experience replay buffer pool, train the policy network and the value network in the second reinforcement learning model, and update the network parameters of the policy network and the value network until the performance of the second reinforcement learning model is stable, thereby obtaining a qualified second reinforcement learning model.

[0137] In this embodiment, by repeatedly executing the above steps S001 to S005, an experience replay buffer pool including a large number of experience tuples is obtained. At this time, a batch of experience tuples are randomly sampled from the experience replay buffer pool to train the policy network and value network in the constructed second reinforcement learning model to update the network parameters of the policy network and value network. After updating the network parameters of the policy network and value network, it is determined whether the performance of the policy network and value network is stable. In the case that the performance of the policy network and value network is not stable, a batch of experience tuples are continuously randomly sampled from the experience replay buffer pool to train the policy network and value network in the constructed second reinforcement learning model to update the network parameters of the policy network and value network. Until the performance of the policy network and value network is stable, a qualified second reinforcement learning model is obtained.

[0138] In combination with the above embodiments, in one implementation, the present application embodiment further provides a method for automatic UI testing. In the method for automatic UI testing, step S003 may include steps 0031 to S0032:

[0139] Step S0031: According to the training sample environment state, an initial execution action is determined through a policy network in a second reinforcement learning model.

[0140] In this embodiment, in order to increase the exploration degree of the robot arm within the range of motion, the present application adds noise to the determined execution action. Specifically: for the training sample environment state input into the constructed second reinforcement learning model, the next action to be executed under the current training sample environment state is selected, and the selection method of this step action is: the initial execution action is determined by the policy network in the second reinforcement learning model.

[0141] Step S0032: adding random noise to the initial execution action to obtain a final execution action.

[0142] In this embodiment, random noise is added to the initial execution action determined by the policy network to obtain a final execution action.

[0143] In combination with the above embodiments, in one implementation, the present application embodiment further provides a method for automatic UI testing. In the method for automatic UI testing, step S006 may include steps S0061 to S0067:

[0144] Step S0061: randomly sample a batch of experience tuples from the experience playback buffer pool, and determine the Q value of each experience tuple through the value network.

[0145] In this embodiment, an experience replay buffer pool including a large number of experience tuples is obtained by repeatedly executing the above steps S001 to S005. At this time, a batch of experience tuples are randomly sampled from the experience replay buffer pool, and the execution action and the training sample environment state are recorded for each experience tuple in the batch of experience tuples. At this time, the Q value of the execution action recorded in each experience tuple is determined through the value network.

[0146] Step S0062: According to the training sample environment state at the next moment in each experience tuple, the target execution action corresponding to each experience tuple is determined through the target strategy network in the second reinforcement learning model.

[0147] In this embodiment, for each training sample environment state at the next moment recorded by each experience tuple in the batch of experience tuples, the target strategy network in the constructed second reinforcement learning model is used to determine the execution action under the training sample environment state at each next moment, and the execution action is the target execution action.

[0148] Step S0063: Based on the determined target execution action corresponding to each experience tuple, the target Q value corresponding to the training sample environment state and the target execution action at the next moment in each experience tuple is calculated through the target value network.

[0149] In this embodiment, for the target execution action under the training sample environment state at each next moment obtained through step S0062, the target Q value corresponding to the training sample environment state and the target execution action at the next moment of each experience tuple is calculated respectively through the target value network in the constructed second reinforcement learning model.

[0150] Step S0064: Update the parameters of the value network based on the difference between the Q value corresponding to the same experience tuple and the target Q value.

[0151] In this embodiment, after obtaining the target Q value under each target action through step S0063, the network parameters of the value network are updated based on the error between the Q value belonging to the same experience tuple (the Q value is the Q value corresponding to the execution action recorded in the same experience tuple) and the target Q value (the target Q value is the Q value calculated based on the target action under the training sample environment state at the next moment in the same experience tuple).

[0152] Step S0065: Update the parameters of the strategy network based on the gradient information of the value network to the action.

[0153] In this embodiment, after each update of the parameters of the value network in step S0064, the network parameters of the strategy network are synchronously updated based on the information of the action gradient of the value network.

[0154] Step S0066: After every second time period, copy the parameters of the policy network to the target policy network, and copy the parameters of the value network to the target value network, so as to update the parameters of the target policy network and the target value network.

[0155] In this embodiment, in order to improve the stability and effect of training, in the process of continuously updating the network parameters of the value network and the policy network, after every second time period, the latest network parameters of the value network are given to the target value network to update the network parameters of the target value network, and the latest network parameters of the policy network are given to the target policy network to update the network parameters of the target policy network. The second time period can be set according to the actual application scenario and is not specifically limited here.

[0156] Step S0067: until the performance of the policy network and the value network in the second reinforcement learning model is stable, a qualified second reinforcement learning model is obtained.

[0157] In this embodiment, after training the value network and the policy network with a batch of randomly sampled experience tuples, the value network and the policy network are trained again with a batch of randomly sampled experience tuples until the performance of the trained value network and the policy network tends to be stable or until the second preset number of trainings is completed, and a qualified second reinforcement learning model is determined to be obtained. At this time, the qualified second reinforcement learning model can be used to determine the optimal robot end stylus speed sequence of the robot end stylus based on the current robot end environment state. The qualified second reinforcement learning model mainly uses the policy network. The second preset number of times can be set according to the actual application scenario and is not specifically limited here.

[0158] In this embodiment, the specific application after the first reinforcement learning model and the second reinforcement learning model are trained and qualified is: Figure 5 As shown, first, after selecting the image data of the target area in the vehicle-machine interface based on the test case, the image data of the sliding box with the same size as the image data of the target area is generated at the starting point L0 of the vehicle-machine interface; secondly, the image data of the vehicle-machine interface, the sliding box and the target area are respectively input into the feature extraction network for feature extraction, and the vehicle-machine interface is set as the environment, the sliding box is set as the intelligent agent, and the target area is set as the target. The next moving action of the intelligent agent is obtained by inputting the extracted features of the three as states into the first reinforcement learning model, and the optimal path to the target area is found by interacting with the environment. Based on the path and the position of the sliding box at the initial state, the position information of the target area in the vehicle interface can be quickly determined; then, the acquired position information of the target area, the range of motion of the robot arm and the environmental information, the current position of the stylus at the end of the robot arm and the current speed of the stylus at the end of the robot arm are input into the second reinforcement learning model as the environmental state, and the optimal control trajectory of the stylus at the end of the robot arm from the starting position to the target area is determined by interacting with the environment, and then the robot arm is used to control the stylus at the end of the robot arm to move to the location of the target area to perform UI interface testing based on the optimal control trajectory; finally, after the stylus at the end of the robot arm reaches the target area to implement the test steps, the vehicle interface asserts the current vehicle interface and the expected vehicle interface and generates a test report. When the assertion fails, the vehicle log is automatically pulled and stored before executing the next test case. When the assertion succeeds, the next test case is directly executed.

[0159] Based on the same inventive concept, an embodiment of the present application provides a system for automatic UI testing, such as Figure 6 As shown, the system 600 includes:

[0160] The feature extraction module 601 is used to extract features of the vehicle-machine interface, the starting sliding frame and the target area to be measured respectively through a feature extraction network to obtain vehicle-machine interface features, sliding frame features and target area features, wherein the sliding frame has the same size as the target area;

[0161] A feature connection module 602 is used to perform feature connection on the vehicle interface feature, the sliding frame feature and the target area feature to obtain a connection feature;

[0162] The target area identification and positioning module 603 is used to determine that the connection feature is the vehicle-machine interface environment state, input it into the first reinforcement learning model for processing, and obtain the location information of the target area in the vehicle-machine interface;

[0163] The motion trajectory determination module 604 is used to input the current robot arm environment state into the second reinforcement learning model for processing to obtain the optimal motion trajectory of the stylus at the end of the robot arm moving to the target area, wherein the current robot arm environment state includes the position information of the target area in the vehicle interface and the spatial position information of the target area in the vehicle;

[0164] The testing module 605 is used to control the end stylus of the robot arm to move to the target area for automatic testing based on the optimal motion trajectory.

[0165] Optionally, the feature extraction module 601 includes:

[0166] The first feature extraction module is used to input the vehicle interface, the starting sliding frame and the target area to be measured into the feature extraction network for feature extraction;

[0167] The second feature extraction module is used to unify the vehicle interface features, sliding box features and target area features to the same scale through the spatial pyramid pooling layer of the feature extraction network.

[0168] Optionally, the motion trajectory determination module 604 includes:

[0169] A data determination module, used to determine the current position of the stylus at the end of the robotic arm and the speed of the stylus at the end of the robotic arm;

[0170] The current robot arm environment state determination module is used to determine the robot arm motion range and environment information, the location information of the target area, the current robot arm end stylus position and the robot arm end stylus speed as the current robot arm environment state;

[0171] an optimal robot arm end stylus speed sequence determination module, used for inputting the current robot arm environment state into the second reinforcement learning model for processing, and obtaining an optimal robot arm end stylus speed sequence for moving the robot arm end stylus to the target area;

[0172] The testing module 605 is used to control the stylus at the end of the robotic arm to move to the target area along the optimal stylus at the end of the robotic arm speed sequence through the robotic arm to perform automatic testing based on the optimal stylus at the end of the robotic arm speed sequence.

[0173] Optionally, the system 600 further includes:

[0174] A first model building module, used to build a first reinforcement learning model;

[0175] A first model training module, used for inputting a training sample environment state into the first reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of a vehicle interface feature, a sliding box feature, and a target area feature;

[0176] A first execution action determination module, used to calculate the Q value of each action through the Q value prediction network in the first reinforcement learning model according to the training sample environment state, and select an execution action based on the action selection strategy;

[0177] A first data determination module, configured to determine the training sample environment state and the immediate reward at the next moment of the training sample environment state based on the selected execution action;

[0178] A first experience tuple determination storage module is used to determine the training sample environment state, the execution action, the immediate reward and the training sample environment state at the next moment as an experience tuple, and store it in an experience playback buffer pool;

[0179] The first model parameter updating module is used to randomly sample a batch of experience tuples from the experience replay buffer pool, train the Q-value prediction network, and update the network parameters of the Q-value prediction network until the performance of the Q-value prediction network is stable, thereby obtaining a first reinforcement learning model that has passed the training.

[0180] Optionally, the first execution action determination module includes:

[0181] A first Q value determination module is used to calculate the Q value of each action through a Q value prediction network algorithm in a first reinforcement learning model according to the training sample environment state;

[0182] The first execution action determination submodule is used to select an execution action from each action according to the Q value of each action, the ε-greedy strategy and the action shielding algorithm.

[0183] Optionally, the first model parameter updating module includes:

[0184] A first best action determination module is used to randomly sample a batch of experience tuples from the experience replay buffer pool, and select the best action under the training sample environment state at each next moment through an ε-greedy strategy for the training sample environment state at the next moment recorded in each experience tuple;

[0185] A first target Q value determination module, used to calculate the target Q value of each optimal action through the target network in the first reinforcement learning model;

[0186] A Q-value prediction network parameter updating module is used to update the parameters of the Q-value prediction network according to the error between the Q-value corresponding to the same experience tuple and the target Q-value;

[0187] A first target network parameter updating module, used for copying the parameters of the Q-value prediction network to the target network after every first time period, so as to update the parameters of the target network;

[0188] The first model determination module is used to obtain a first reinforcement learning model that has passed the training until the performance of the Q-value prediction network is stable.

[0189] Optionally, the system 600 further includes:

[0190] A second model building module, used to build a second reinforcement learning model;

[0191] A second model training module is used to input the training sample environment state into the first reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of the robot arm motion range and environment information, the position information of the target area, the robot arm end stylus position and the robot arm end stylus speed;

[0192] A second execution action determination module, used to determine an execution action through a policy network in a second reinforcement learning model according to the training sample environment state;

[0193] A second data determination module, configured to determine the training sample environment state and the immediate reward at the next moment of the training sample environment state based on the determined execution action;

[0194] A second experience tuple determination storage module is used to determine the training sample environment state, the execution action, the immediate reward and the training sample environment state at the next moment as an experience tuple, and store it in an experience playback buffer pool;

[0195] The second model parameter updating module is used to randomly sample a batch of experience tuples from the experience replay buffer pool, train the policy network and the value network in the second reinforcement learning model, and update the network parameters of the policy network and the value network until the performance of the second reinforcement learning model is stable, thereby obtaining a qualified second reinforcement learning model.

[0196] Optionally, the second execution action determination module includes:

[0197] An initial execution action determination module, used to determine the initial execution action through the policy network in the second reinforcement learning model according to the training sample environment state;

[0198] The noise adding module is used to add random noise to the initial execution action to obtain the final execution action.

[0199] Optionally, the second model parameter updating module includes:

[0200] A second Q value determination module is used to randomly sample a batch of experience tuples from the experience playback buffer pool, and determine the Q value of each experience tuple through a value network;

[0201] A second execution action determination submodule is used to determine the target execution action corresponding to each experience tuple through the target strategy network in the second reinforcement learning model according to the training sample environment state at the next moment in each experience tuple;

[0202] A second target Q value determination module is used to calculate the target Q value corresponding to the training sample environment state and the target execution action in each experience tuple at the next moment through the target value network based on the target execution action corresponding to each determined experience tuple;

[0203] A value network parameter updating module, used to update the parameters of the value network based on the difference between the Q value corresponding to the same experience tuple and the target Q value;

[0204] The policy network parameter update module is used to update the parameters of the policy network based on the gradient information of the value network to the action;

[0205] A second target network parameter updating module, used to copy the parameters of the policy network to the target policy network and the parameters of the value network to the target value network after every second time period, so as to update the parameters of the target policy network and the target value network;

[0206] The second model determination module is used to obtain a qualified second reinforcement learning model until the performance of the policy network and the value network in the second reinforcement learning model is stable.

[0207] Based on the same inventive concept, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and running on the processor. When the computer program is executed by the processor, it implements the steps in a method for automatic UI testing as described in the first aspect of the present application.

[0208] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in a method for automatic UI testing as described in the first aspect of the present application are implemented.

[0209] As for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0210] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0211] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0212] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0213] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0214] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0215] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0216] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0217] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0218] The above is a detailed introduction to the method, system and product for automatic testing of UI provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for automatic UI testing, characterized in that: The method comprises: The vehicle interface, the starting sliding frame and the target area to be measured are respectively extracted by a feature extraction network to obtain vehicle interface features, sliding frame features and target area features, wherein the sliding frame has the same size as the target area; Performing feature connection on the vehicle machine interface feature, the sliding frame feature and the target area feature to obtain a connection feature; Determine the connection feature as a vehicle-machine interface environment state, input it into a first reinforcement learning model for processing, and obtain position information of the target area in the vehicle-machine interface; Inputting the current robot arm environment state into the second reinforcement learning model for processing to obtain the optimal motion trajectory of the stylus at the end of the robot arm to the target area, wherein the current robot arm environment state includes the position information of the target area in the vehicle interface and the spatial position information of the target area in the vehicle; Based on the optimal motion trajectory, the end stylus of the robotic arm is controlled to move to the target area for automatic testing.

2. A method for automatic UI testing according to claim 1, characterized in that: The feature extraction network is used to extract features of the vehicle interface, the starting sliding frame and the target area to be measured, respectively, to obtain the vehicle interface features, the sliding frame features and the target area features, including: The vehicle computer interface, the starting sliding frame and the target area to be measured are input into the feature extraction network for feature extraction; The vehicle interface features, sliding box features and target area features are unified to the same scale through the spatial pyramid pooling layer of the feature extraction network.

3. A method for automatic UI testing according to claim 1, characterized in that: The current robot arm environment state is input into the second reinforcement learning model for processing to obtain the optimal motion trajectory of the stylus at the end of the robot arm moving to the target area, including: Determine the current end-of-arm stylus position and end-of-arm stylus speed; Determine the robot arm motion range and environmental information, the location information of the target area, the current robot arm end stylus position and the robot arm end stylus speed as the current robot arm environment state; Inputting the current robot arm environment state into a second reinforcement learning model for processing to obtain an optimal robot arm end stylus velocity sequence for moving the robot arm end stylus to the target area; The step of controlling the end stylus of the robotic arm to move to the target area for automatic testing based on the optimal motion trajectory includes: Based on the optimal robot arm end stylus speed sequence, the robot arm controls the robot arm end stylus to move along the optimal robot arm end stylus speed sequence to the target area for automatic testing.

4. A method for automatic UI testing according to claim 1, characterized in that: The method further comprises: Build the first reinforcement learning model; Inputting a training sample environment state into the first reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of vehicle-machine interface features, sliding box features, and target area features; According to the training sample environment state, the Q value of each action is calculated by the Q value prediction network in the first reinforcement learning model, and the execution action is selected based on the action selection strategy; Based on the selected execution action, determining the training sample environment state and the immediate reward at the next moment of the training sample environment state; Determine the training sample environment state, the execution action, the immediate reward and the training sample environment state at the next moment as an experience tuple, and store it in an experience playback buffer pool; A batch of experience tuples are randomly sampled from the experience replay buffer pool, the Q-value prediction network is trained, and the network parameters of the Q-value prediction network are updated until the performance of the Q-value prediction network is stable, thereby obtaining a first reinforcement learning model that has passed the training.

5. A method for automatic UI testing according to claim 4, characterized in that: According to the training sample environment state, the Q value of each action is calculated by the Q value prediction network in the first reinforcement learning model, and the execution action is selected based on the action selection strategy, including: According to the training sample environment state, the Q value of each action is calculated by the Q value prediction network algorithm in the first reinforcement learning model; Determine a non-preferred action in each of the actions based on a priori rules, and assign a Q value less than a set threshold to the determined non-preferred action; According to the Q value of each action, the ε-greedy strategy selects an action to be executed from each action.

6. A method for automatic UI testing according to claim 4, characterized in that: Randomly sampling a batch of experience tuples from the experience replay buffer pool, training the Q-value prediction network, and updating the network parameters of the Q-value prediction network until the performance of the Q-value prediction network is stable, thereby obtaining a first reinforcement learning model that has passed the training, including: A batch of experience tuples are randomly sampled from the experience replay buffer pool, and for the training sample environment state at the next moment recorded in each experience tuple, the best action under the training sample environment state at each next moment is selected through an ε-greedy strategy, wherein the best action is the action corresponding to the maximum Q value obtained by calculating the Q value of each action through the target network in the first reinforcement learning model; updating the parameters of the Q-value prediction network according to the error between the Q-value corresponding to the same experience tuple and the target Q-value, wherein the target Q-value is the Q-value of the optimal action; After every first time period, copying the parameters of the Q-value prediction network to the target network to update the parameters of the target network; Until the performance of the Q-value prediction network is stabilized, a first reinforcement learning model that has passed the training is obtained.

7. A method for automatic UI testing according to claim 1, characterized in that: The method further comprises: Build a second reinforcement learning model; Inputting the training sample environment state into the second reinforcement learning model for model training, wherein the training sample environment state is an environment state composed of the robot arm motion range and environment information, the position information of the target area, the robot arm end stylus position and the robot arm end stylus speed; Determining an execution action through a policy network in a second reinforcement learning model according to the training sample environment state; Based on the determined execution action, determining the training sample environment state and the immediate reward at the next moment of the training sample environment state; Determine the training sample environment state, the execution action, the immediate reward and the training sample environment state at the next moment as an experience tuple, and store it in an experience playback buffer pool; A batch of experience tuples are randomly sampled from the experience replay buffer pool, the policy network and the value network in the second reinforcement learning model are trained, and the network parameters of the policy network and the value network are updated until the performance of the second reinforcement learning model is stable, thereby obtaining a qualified second reinforcement learning model.

8. A method for automatic UI testing according to claim 7, characterized in that: According to the training sample environment state, determining the execution action through the policy network in the second reinforcement learning model includes: Determining an initial execution action through a policy network in a second reinforcement learning model according to the training sample environment state; Random noise is added to the initial execution action to obtain a final execution action.

9. A method for automatic UI testing according to claim 7, characterized in that: Randomly sampling a batch of experience tuples from the experience replay buffer pool, training the policy network and the value network in the second reinforcement learning model, updating the network parameters of the policy network and the value network until the performance of the second reinforcement learning model is stable, and obtaining a trained second reinforcement learning model, including: Randomly sampling a batch of experience tuples from the experience replay buffer pool, and determining the Q value of each experience tuple through the value network; According to the training sample environment state at the next moment in each experience tuple, the target execution action corresponding to each experience tuple is determined through the target strategy network in the second reinforcement learning model; Based on the target execution action corresponding to each determined experience tuple, the target Q value corresponding to the training sample environment state and the target execution action at the next moment in each experience tuple is calculated through the target value network; Update the parameters of the value network based on the difference between the Q value corresponding to the same experience tuple and the target Q value; Update the parameters of the policy network based on the gradient information of the value network to the action; After every second time period, copy the parameters of the policy network to the target policy network, and copy the parameters of the value network to the target value network, so as to update the parameters of the target policy network and the target value network; Until the performance of the policy network and the value network in the second reinforcement learning model is stable, a qualified second reinforcement learning model is obtained.

10. A UI automatic testing system, characterized in that: The system comprises: A feature extraction module is used to extract features of the vehicle interface, the starting sliding frame and the target area to be measured respectively through a feature extraction network to obtain vehicle interface features, sliding frame features and target area features, wherein the sliding frame has the same size as the target area; A feature connection module, used for performing feature connection on the vehicle interface feature, the sliding frame feature and the target area feature to obtain a connection feature; A target area recognition and positioning module, used to determine that the connection feature is a vehicle-machine interface environment state, input it into a first reinforcement learning model for processing, and obtain the location information of the target area in the vehicle-machine interface; a motion trajectory determination module, used to input the current robot arm environment state into the second reinforcement learning model for processing, and obtain the optimal motion trajectory of the stylus at the end of the robot arm moving to the target area, wherein the current robot arm environment state includes the position information of the target area in the vehicle interface and the spatial position information of the target area in the vehicle; The testing module is used to control the end stylus of the robot arm to move to the target area for automatic testing based on the optimal motion trajectory.

11. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and running on the processor, wherein when the computer program is executed by the processor, the steps in the method for automatic UI testing as described in any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method for automatic UI testing according to any one of claims 1 to 9 are implemented.

Images