Automatic testing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses an automatic testing method and device, equipment and a storage medium. The method comprises the steps of determining a target control and a target state corresponding to the target control based on a to-be-tested scene; obtaining an original state of a target control in the current visual interface, and determining an automatic operation instruction of the target control based on the original state and the target state; adjusting the state of the target control based on the automatic control instruction to obtain an adjusted visual interface, and extracting an actual state corresponding to the target control in the adjusted visual interface; and if the actual state is matched with the target state, determining that the test of the target control passes. According to the embodiment of the invention, the consistency and accuracy of each test can be ensured by accurately controlling the test condition and the execution process, so that the reliability of the test result is improved.

Description

Technical Field

[0001] The present application relates to the technical field of communication system testing, and in particular to an automated testing method, device, electronic device, and computer-readable storage medium. Background Art

[0002] With the development of science and technology, computer vision technology and automated testing systems have been widely used in various fields. In the field of automotive electronic equipment, automated control of CANoe Panel (a visual control set) for testing is a common testing method. This method mainly simulates user operations to test the functions and performance of automotive electronic equipment. However, this method requires a lot of manual operations, is inefficient, and prone to errors. Therefore, in the prior art, there is an urgent need for a method that can automatically and effectively perform automated testing to meet the needs of actual test applications. Summary of the invention

[0003] To solve the above technical problems, the embodiments of the present application provide an automated testing method and device, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] According to one aspect of an embodiment of the present application, there is provided an automated testing method, comprising: determining a target control and a target state corresponding to the target control based on a scenario to be tested; obtaining an original state of the target control in a current visual interface, and determining an automated operation instruction for the target control based on the original state and the target state; adjusting the state of the target control based on the automated control instruction to obtain an adjusted visual interface, and extracting an actual state corresponding to the target control in the adjusted visual interface; if the actual state matches the target state, determining that the test of the target control has passed.

[0005] According to one aspect of an embodiment of the present application, the method also includes: performing image detection on the visual interface based on a target detection model to obtain an original detection result; if the confidence of the original detection result reaches above a preset confidence threshold, determining the original state of the target control based on the detection result, the original state including the original center point coordinates, the original border and the original rotation angle of the target control.

[0006] According to one aspect of an embodiment of the present application, the method also includes: determining the target center point coordinates, target border and target rotation angle corresponding to the target control based on the scene to be tested; determining multiple operation values ​​based on the target center point coordinates, the target border and the target rotation angle and the original center point coordinates, the original border and the original rotation angle; generating multiple operations corresponding to the drive-level operation based on the multiple operation values, and generating the automated operation instructions based on the multiple operations.

[0007] According to one aspect of an embodiment of the present application, the method also includes: performing image detection on the adjusted visual interface based on the target detection model to obtain an adjusted detection result; if the confidence of the adjusted detection result reaches above a preset confidence threshold, determining the actual state corresponding to the target control based on the adjusted detection result.

[0008] According to one aspect of an embodiment of the present application, the method also includes: obtaining a control image set in the visualization interface, and annotating the control based on the position of the control on the visualization interface and the function of the control to obtain a label corresponding to the control; generating a control image label set based on the control image set and the label corresponding to the control, the control image label set including the center point coordinates, border and rotation angle corresponding to the control; and obtaining the target detection model by training based on the control image label set.

[0009] According to one aspect of an embodiment of the present application, the method further includes: parsing the scenario to be tested to obtain key functions corresponding to the scenario to be tested; determining corresponding control tags based on the key functions to search for matching target controls based on the control tags.

[0010] According to one aspect of an embodiment of the present application, the method also includes: determining the expected result corresponding to the scenario to be tested; after the execution of the automation control instruction is completed, obtaining the operation result corresponding to the target control; if the operation result matches the expected result, determining that the test of the target control has passed.

[0011] According to one aspect of an embodiment of the present application, an automated testing device is provided, comprising: a determination module, for determining a target control and a target state corresponding to the target control based on a scenario to be tested; an acquisition module, for acquiring the original state of the target control in a current visual interface, and determining an automated operation instruction for the target control based on the original state and the target state; an adjustment module, for adjusting the original state based on the automated control instruction, obtaining an adjusted visual interface, and extracting an actual state corresponding to the target control in the adjusted visual interface; and a judgment module, for determining that the test of the target control has passed if the actual state matches the target state.

[0012] According to one aspect of the embodiments of the present application, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the automated testing method as described above. According to one aspect of the embodiments of the present application, there is provided a computer-readable storage medium, on which computer-readable instructions are stored, which, when executed by a processor of a computer, enables the computer to execute the automated testing method as described above.

[0013] According to one aspect of an embodiment of the present application, a computer program product is also provided, including a computer program, which implements the steps in the automated testing method described above when executed by a processor.

[0014] In the technical solution provided in the embodiments of the present application, the target control and the target state corresponding to the target control are determined for the test scenario, and then the original state corresponding to the target control in the real-time visual interface is obtained, and then the automated operation instructions of the target control are determined according to the original state and the target state of the target control. Therefore, automated testing can greatly reduce the time and labor cost of manual testing. Through the pre-defined target control and target state, and the automatically generated and executed operation instructions, the test process can be executed quickly and accurately, significantly improving the test efficiency. Then, the state of the target control is adjusted through automated control instructions to obtain the adjusted visual interface, and then the actual state corresponding to the target control in the adjusted visual interface is extracted. When the actual state matches the target state, it can be determined that the test of the target control has passed. Automated testing reduces errors caused by human factors, such as missing test points, misjudgment of test results, etc. By precisely controlling the test conditions and execution process, the consistency and accuracy of each test can be ensured, thereby improving the reliability of the test results.

[0015] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0017] Figure 1 is a schematic diagram of a visualization interface of automated testing shown in an exemplary embodiment of the present application;

[0018] Figure 2 is a schematic diagram of an implementation environment for performing automated testing shown in an exemplary embodiment of the present application;

[0019] Figure 3 is a flow chart of an automated testing method shown in an exemplary embodiment of the present application;

[0020] Figure 4 is a flow chart of an automated testing method shown in another exemplary embodiment of the present application;

[0021] Figure 5 is a flow chart of an automated testing method shown in another exemplary embodiment of the present application;

[0022] Figure 6 is a flow chart of an automated testing method shown in another exemplary embodiment of the present application;

[0023] Figure 7 is a flow chart of an automated testing method shown in another exemplary embodiment of the present application;

[0024] Figure 8 is a flow chart of an automated testing method shown in another exemplary embodiment of the present application;

[0025] Fig. 9 is a flow chart of an automated testing method shown in another exemplary embodiment of the present application;

[0026] Fig.10 It is a brief flow chart of automated testing in an exemplary application scenario;

[0027] Fig.11 is a block diagram of an automated testing device shown in an exemplary embodiment of the present application;

[0028] Fig.12 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0030] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0031] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0032] The term "multiple" as used in this application refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0033] First of all, it should be noted that CANoe Panel is a very powerful feature in CANoe software, which allows users to create a custom graphical user interface (GUI) for interacting with a simulated or actual running network environment. This interface can simulate various physical controls, such as buttons, switches, knobs, dashboards, etc., and directly map the operations of these controls to signals in network messages (such as CAN, LIN, FlexRay, etc.). Through CANoe Panel, testers can easily simulate user operations, send control commands to ECU (electronic control unit) or other network nodes, and observe the response of the system. This interactive method greatly simplifies the test process, allowing testers to more intuitively understand the behavior of the system and quickly identify potential problems.

[0034] In CANoe, Panel is a graphical user interface (GUI) that allows users to interact with simulated or actual running ECU (electronic control unit) in an intuitive way. Panel can be designed to simulate various switches, buttons, indicator lights, dashboards, etc. These elements can be directly mapped to signals in network messages. In this way, testers can simulate user operations, send control commands, and observe the system's response.

[0035] Please refer to the figure Figure 1 , Figure 1 is a schematic diagram of a visualization interface shown in an exemplary embodiment of the present application, and it can be seen that Figure 1 It includes controls, as well as the center point coordinates, border height, width, and rotation angle corresponding to the space. Among them, "control image" may refer to the icon or representation of the control) The content is very rich. These controls are used to create graphical user interfaces, and there are Boolean switches. This type of control is usually used to indicate the switch state, such as on or off. They can have a variety of styles, including button-type, sliding-type, etc., and when clicked, they may be accompanied by icon or color changes to indicate state changes. Buttons, standard button controls, are used to trigger an operation or command. Text or icons are usually displayed on buttons to indicate their functions. And text boxes that allow users to enter data or display data. Users can enter numbers, text, etc.

[0036] Figure 2 It is a schematic diagram of the implementation environment for performing automated testing shown in an exemplary embodiment of the present application. As shown in 2, in the automated testing scenario, the server 220 first determines the scenario to be tested, and then the server 220 determines at least one target control corresponding to the scenario to be tested and a target state corresponding to at least one target control from the scenario to be tested. Then the server 220 obtains the original state of the target control on the visual interface of the smart terminal 210, and then can determine the automated operation instructions of the target control according to the original state and the target state, and then adjusts the state of the target control based on the automated control instructions to obtain the adjusted visual interface, and extracts the actual state corresponding to the target control in the adjusted visual interface; if the actual state matches the target state, it is determined that the test of the target control has passed.

[0037] in, Figure 2 The smart terminal 210 shown can be any terminal device that supports the installation of navigation map software, such as a smart phone, a car computer, a tablet computer, a laptop computer, or a wearable device, but is not limited thereto. Figure 2The guide server 220 shown can be, for example, an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, and is not limited here. The smart terminal 210 can communicate with the server 220 through wireless networks such as 3G (third generation mobile information technology), 4G (fourth generation mobile information technology), and 5G (fifth generation mobile information technology), and is not limited here.

[0038] With the development of science and technology, computer vision technology and automated testing systems have been widely used in various fields. In the field of automotive electronic equipment, automated control of CANoe Panel (a visual control set) for testing is a common testing method. This method mainly simulates user operations to test the functions and performance of automotive electronic equipment. However, this method requires a lot of manual operations, is inefficient, and prone to errors. Therefore, in the prior art, there is an urgent need for a method that can automatically and effectively perform automated testing to meet the needs of actual test applications.

[0039] The above-mentioned problems are generally applicable. To solve these problems, the embodiments of the present application respectively provide an automated testing method, an automated testing device, an electronic device, a computer-readable storage medium, and a computer program product, which will be described in detail below.

[0040] like Figure 3 As shown, in an exemplary embodiment, the automated testing method includes at least steps S310 to S340, which are described in detail as follows:

[0041] Step S310: determining a target control and a target state corresponding to the target control based on the scenario to be tested.

[0042] Specifically, a test scenario is a series of steps and conditions that describe the interaction between the user and the system in a specific test environment. A clearly defined test scenario should include: Scenario name: briefly describe the theme or purpose of the test scenario; Preconditions: conditions that must be met before executing the test scenario, such as user login status, system configuration, etc.; Steps: detailed list of operations performed by the user, including interactions with interface controls; Input data: data that needs to be input during the test, such as text, files, commands, etc.; Expected results: expected output or system response for each step or the entire scenario. After clarifying the scenario to be tested, it is necessary to analyze the key controls involved in the scenario. Controls are interface elements for users to interact with applications, such as buttons, text boxes, lists, etc.

[0043] Exemplarily, the test scenario based on CANoe Panel is planned, for example, the door unlocking is tested, the opening and closing of the lights is tested, etc., the unlocking or locking signal is sent through the CANoe Panel, and the door can be verified to respond correctly and feedback the status or the light switch is controlled through the CANoe Panel to check whether the lighting and extinguishing of the lights (such as low beam, high beam, turn signal, etc.) are in line with expectations. Specifically, the target controls included in the door unlocking test may include an unlock button, a lock button, a status indicator light, etc., and the target controls for the opening and closing test of the lights include a high beam switch and a low beam switch. Then, the target state corresponding to the target control is determined according to the scenario to be tested, and then, an existing CANoe project is created or opened, the relevant DBC file and network environment are configured, and the Panel control is mapped to the signal in the DBC file to ensure that the control operation can correctly send the corresponding signal to the vehicle ECU. Connect the CANoe device to the actual vehicle to ensure normal communication. Operate the control through the Panel interface to observe whether the vehicle response is in line with expectations.

[0044] Step S320, obtaining the original state of the target control in the current visual interface, and determining the automation operation instruction of the target control based on the original state and the target state.

[0045] Specifically, it is necessary to be able to access and read the current state of the target control in the current visual interface (such as CANoe Panel or other GUI interface). This can usually be achieved in the following ways: using graphical user interface automation tools: such as Selenium (mainly used for Web interfaces), PyAutoGUI (for desktop applications) or CANoe-specific automation tools (if available); directly through the interface or API: if CANoe provides an API to access the status of Panel controls, then these APIs can be called directly to obtain the status; screen reading technology: for GUIs that do not support direct API access, screenshots and image recognition technology can be used to identify the status of controls.

[0046] Compare the acquired original state with the target state to determine whether an automated operation needs to be performed to change the state of the control, where the target state is the state you want the control to reach during the test or control process. This is usually predefined and is part of the test script or automated control logic. If the original state is inconsistent with the target state, then the corresponding automated operation instructions need to be generated to change the state of the control. These instructions may include: click: if the control is a button, you may need to simulate a mouse click to activate it; enter text: if the control is a text box, you may need to enter specific text; send a signal: in the context of CANoe, if the control is associated with an ECU signal, you may need to send the corresponding CAN signal through a CAPL script or other mechanism; change the attribute value: if the state of the control is controlled by changing the value of one of its attributes (such as the selected state of a check box), you need to find the corresponding method or function to change this attribute value.

[0047] Step S330, adjusting the state of the target control based on the automated control instruction, obtaining an adjusted visual interface, and extracting the actual state corresponding to the target control in the adjusted visual interface.

[0048] Specifically, first, it is necessary to execute the automated control instructions generated in the previous embodiment. These instructions may include simulating user operations (such as clicking, inputting, etc.), or directly sending signals to the system through an API. In the context of CANoe, this usually involves using a CAPL script or other mechanism to send a CAN signal to the vehicle ECU, thereby triggering a change in the control state. After executing the control instruction, it takes some time to respond and update the control state on the interface. Therefore, it is necessary to add an appropriate waiting mechanism in the automated test script. This can be a simple fixed time delay or a more intelligent waiting condition, such as checking whether a state variable has been changed or whether an event has been triggered. Once the waiting mechanism confirms that the state has changed, the adjusted visual interface can be obtained.

[0049] In automated testing, this usually means capturing a screenshot of the current GUI, or using a GUI automation tool to read the current state of the controls on the interface. From the adjusted visual interface, you need to extract the actual state of the target control. This can be achieved in a variety of ways, including but not limited to: Screen image analysis: Use image recognition technology to analyze screenshots and determine the state of controls (such as whether a button is pressed, whether an indicator light is on, etc.); GUI automation tools: Use GUI automation tools such as Selenium and PyAutoGUI to query the current property values ​​of controls; CANoe API or CAPL script: If CANoe provides a corresponding API or can access the control state through CAPL scripts, then directly call these interfaces to obtain status information.

[0050] Step S340: If the actual state matches the target state, it is determined that the test of the target control passes.

[0051] Specifically, when the actual state matches the target state, it can be determined that the test for the target control has passed. In the automated test framework, once the test steps are executed and the necessary state changes are waited for, the actual state of the target control is extracted and compared with the preset target state. If the two are consistent, the test is considered to have passed successfully; if they are inconsistent, the test fails and further analysis and debugging may be required to find out the problem. For tools such as CANoe, you can implement this process by writing CAPL scripts. CAPL scripts can simulate user input, send CAN messages, read ECU responses, perform conditional judgments, and record test results. With a well-designed CAPL script, you can automatically execute complex test scenarios to verify whether the behavior of various controls and systems in the vehicle network meets expectations. When the test passes, information about the success of the test is usually recorded, and the results may be reported to relevant stakeholders. If the test fails, the cause of the failure and context information need to be recorded in detail for subsequent troubleshooting and repair work.

[0052] In this embodiment, the target control and the target state corresponding to the target control are determined for the test scenario, and then the original state corresponding to the target control in the real-time visual interface is obtained, and then the automated operation instructions of the target control are determined according to the original state and the target state of the target control. Therefore, automated testing can greatly reduce the time and labor cost of manual testing. Through the pre-defined target control and target state, and the automatically generated and executed operation instructions, the test process can be executed quickly and accurately, which significantly improves the test efficiency. Then, the state of the target control is adjusted through automated control instructions to obtain the adjusted visual interface, and then the actual state corresponding to the target control in the adjusted visual interface is extracted. When the actual state matches the target state, it can be determined that the test of the target control has passed. Automated testing reduces errors caused by human factors, such as missing test points, misjudgment of test results, etc. By accurately controlling the test conditions and execution process, the consistency and accuracy of each test can be ensured, thereby improving the reliability of the test results.

[0053] Further, based on the above embodiments, please refer to Figure 4 In one of the exemplary embodiments provided in the present application, the specific implementation process of the above-mentioned automated testing method may further include step S410 and step S420, which are described in detail as follows:

[0054] Step S410, performing image detection on the visual interface based on the target detection model to obtain an original detection result;

[0055] Step S420, if the confidence of the original detection result reaches above a preset confidence threshold, the original state of the target control is determined based on the detection result, and the original state includes the original center point coordinates, the original border and the original rotation angle of the target control.

[0056] It should be noted that the target detection model can be a yolov5 model, where the yolov5 model is an object detection task designed for a computer vision model. It is an upgraded version of the YOLO series and is famous for its real-time object detection capability.

[0057] Specifically, before inputting the image into the target detection model, some preprocessing steps may be required, such as resizing the image, normalizing, color space conversion, etc., to ensure that the model can process the input image efficiently and accurately. Use a pretrained or customized target detection model to detect the preprocessed image. The model identifies the target objects in the image (target controls in this context) and outputs a bounding box and a corresponding confidence score for each detected object. The bounding box is usually represented as the position and size of the target control in the image, while the confidence score indicates the model's confidence in the detection result. For each detected target control, check whether its confidence score reaches the preset confidence threshold. This threshold is an adjustable parameter used to balance the accuracy and recall of the detection. If the confidence score is higher than the threshold, the detection result is considered reliable.

[0058] Specifically, the original state of the target control can be extracted from the current visual interface through the target detection model, where the original state includes: original center point coordinates: calculate the center point coordinates of the bounding box, which can be obtained by taking the average of the coordinates of the upper left corner and the lower right corner of the bounding box; original border: directly use the bounding box output by the model as the original border of the target control. The bounding box is usually composed of four coordinates (upper left corner x, y and lower right corner x, y), or equivalently, represented by the center point coordinates, width and height; original rotation angle: if the target control is rotated in the image and the detection model supports the detection of rotated objects, the rotation angle can be directly obtained from the output of the model. However, if the model does not support rotation detection, it may be necessary to estimate the rotation angle through other methods (such as image processing technology). In some cases, if the rotation of the target control has little effect on subsequent processing, the rotation angle can also be ignored.

[0059] In this embodiment, the target detection model can be used to accurately identify the position, shape, and orientation of controls, thereby greatly improving the accuracy and efficiency of testing. By accurately identifying the control status of the user interface, automated testing can discover problems in interface layout, control functionality, etc. earlier.

[0060] Further, based on the above embodiment, please refer to Figure 5 In one of the exemplary embodiments adopted in this application, the specific implementation process of the above-mentioned automated testing method may also include steps S510 to S530, which are described in detail as follows:

[0061] Step S510, determining the target center point coordinates, target border, and target rotation angle corresponding to the target control based on the scene to be tested.

[0062] Specifically, it is necessary to analyze the scene to be tested in detail to understand the interface elements, control layout, and interaction relationship between controls contained therein. This helps to determine which controls are the target controls that need to be focused on, as well as the target center point coordinates, target border, and target rotation angle corresponding to the target state of the target control.

[0063] Step S520 , determining a plurality of operation values ​​based on the target center point coordinates, the target bounding box, the target rotation angle, and the original center point coordinates, the original bounding box, and the original rotation angle.

[0064] Specifically, the moving distance and direction of the center point of the target control relative to the center point of the original control are calculated to obtain the corresponding operation value, and the operation value can be a two-dimensional vector, which represents the movement amount in the X-axis and Y-axis directions (i.e., the target center point coordinates minus the original center point coordinates). This vector indicates the direction and distance of the control movement. Evaluate the change in the border size of the target control relative to the border size of the original control. Operation value: usually includes two values, one representing the change in border width (target width minus original width), and the other representing the change in border height (target height minus original height). These values ​​can be positive (indicating expansion), negative (indicating contraction), or zero (indicating no change). Operation value of rotation angle. Measure the change in the rotation angle of the target control relative to the original control. Operation value: is an angle value, which represents the change from the original rotation angle to the target rotation angle (target rotation angle minus original rotation angle). This value can be positive (indicating clockwise rotation), negative (indicating counterclockwise rotation), or zero (indicating no rotation change). Note that you may want to account for periodicity when dealing with angles (e.g., 360 degrees equals 0 degrees), but in practice you are usually only interested in smaller angle changes.

[0065] Moreover, these operation values ​​together describe the geometric transformation and property changes of the target control relative to its original state. They can be used in a variety of scenarios, such as state verification in automated testing, iteration tracking of UI design, or user interaction analysis in dynamic interfaces. By comparing these operation values, we can quickly identify the specific changes of the control on the interface and make corresponding processing or decisions accordingly.

[0066] Step S530: generating multiple operations corresponding to the driving-level operations based on the multiple operation values, and generating automatic operation instructions based on the multiple operations.

[0067] Specifically, multiple operation values ​​obtained from the control state comparison are parsed, including the offset of the center point coordinates, the change of the border size, the change of the rotation angle, etc. The parsed operation values ​​are mapped to specific driver-level operations. This usually involves calling the UI framework or automated testing tool API.

[0068] Center point coordinate offset: may be converted into a mouse move operation to move the mouse pointer to the new center point position of the target control.

[0069] Border size change: It may be necessary to combine the interaction of specific UI elements (such as scroll bars, dividers, etc.) to simulate size changes, or in some cases, this may be directly reflected as a change in content without explicit operation. However, if the size is changed by dragging the edge of the control, it can be converted into a simulated mouse drag operation.

[0070] Rotation angle change: In standard web or desktop applications, the rotation of controls is usually not the result of direct user operation, but driven by the logic or animation effects within the application. However, in some specific test environments (such as game testing, graphical interface testing), it may be necessary to simulate keyboard input (such as specific shortcut keys), use the rotation function provided by the test tool, or call the underlying graphics API to achieve this.

[0071] Then, based on the mapped driver-level operations, specific automation operation instructions are generated. These instructions will guide the automated testing tool to perform the corresponding operations. Use the API provided by the test framework or tool to write code or scripts that will perform the above-mentioned mouse movement, click, drag or keyboard input operations. Set necessary waiting times or checkpoints to ensure the correct order and dependencies between operations.

[0072] In this embodiment, by accurately calculating the center point coordinates, border and rotation angle of the target control, it can be ensured that the test operation is performed directly on the target control to avoid misoperation of other controls. The method of generating automated operation instructions based on the state changes of the target control has significant beneficial effects in improving test accuracy, efficiency, flexibility and reducing maintenance costs.

[0073] Further, based on the above embodiments, please refer to Figure 6 In one of the exemplary embodiments provided in the present application, the specific implementation process of the above-mentioned automated testing method may further include step S610 and step S620, which are described in detail as follows:

[0074] Step S610: performing image detection on the adjusted visualization interface based on the target detection model to obtain an adjusted detection result.

[0075] Specifically, following the above-mentioned embodiments, the adjusted visual interface can be subjected to image detection through a trained target detection model to detect the target controls in the adjusted visual interface and the corresponding states of the target controls, and the adjusted visual interface can be subjected to image detection to obtain detection results corresponding to the target controls in the adjusted visual interface.

[0076] Step S620: If the confidence of the adjusted detection result reaches above a preset confidence threshold, the actual state corresponding to the target control is determined based on the adjusted detection result.

[0077] Specifically, the target control is initially detected by a certain detection method (such as image recognition, OCR technology, UI automation test framework, etc.) to obtain its current state (such as whether it is visible, enabled, selected, etc.). A confidence assessment is performed on the adjusted detection result. Confidence is an indicator to measure the reliability of the detection result, which is usually derived based on a series of statistical data and algorithms. The level of confidence reflects the degree of consistency between the detection result and the actual state. In order to ensure the reliability of the detection result, a confidence threshold is preset. This threshold is determined according to actual needs and application scenarios, and is usually a value between 0 and 1 (for example, 0.9 represents 90% confidence). If the confidence of the adjusted detection result reaches above the preset confidence threshold, then it can be considered that the detection result is reliable enough to serve as a basis for determining the actual state of the target control. In this case, the actual state of the target control will be updated or confirmed based on the adjusted detection result.

[0078] In this embodiment, image detection is performed on the adjusted visual interface through the target detection model, and the actual state of the target control is determined according to the confidence of the adjusted detection result, which has significant beneficial effects in improving the accuracy, real-time, reliability and stability of the test, while reducing the test cost and improving the test efficiency.

[0079] Further, based on the above embodiments, please refer to Figure 7 In one of the exemplary embodiments provided in the present application, the specific implementation process of the above-mentioned automated testing method may further include steps S710 to S730, which are described in detail as follows:

[0080] Step S710, obtaining a set of control images in the visualization interface, and annotating the controls based on their positions on the visualization interface and their functions to obtain labels corresponding to the controls.

[0081] Specifically, you can obtain a training set of control images in the panel control panel to be trained. First, you need to collect a large number of control images of the CANoe Panel control panel, such as Picture Box, Switch / Indicator, LEDControl, etc. These images will be used as training sets for training the target detection model. Then, use the labeling tool to label the images, determine the meaning of the images, the position of the controls on the visual interface, and the functions of the controls. For example, you can use labeling tools (such as rolabelimg) to annotate the images in the training set. Each control needs to be labeled to determine its position and function in the image.

[0082] Step S720: Generate a control image label set based on the control image set and the label corresponding to the control, wherein the control image label set includes the center point coordinates, border, and rotation angle corresponding to the control.

[0083] First, if the control images are not of the same size, the control images need to be normalized to obtain a uniform size. For controls of regular shapes (such as rectangles and circles), the center point coordinates can be calculated from the boundary coordinates of the control. For example, the center point coordinates of a rectangular control can be calculated from the boundary coordinates to obtain the midline point coordinates. If the control shape is irregular, more complex image processing methods may be required to estimate the center point, such as contour-based centroid calculation methods. Then, the border is usually defined by the outermost boundary coordinates of the control. For rectangular controls, the corresponding boundary coordinates can be directly determined. For non-rectangular controls, the border may need to be identified by edge detection algorithms (such as the Canny edge detector) or contour detection algorithms (such as the Hough transform). In addition, if the control image is rotated, its rotation angle needs to be detected and recorded. This can be achieved by calculating the angle between the main axis of the control and the image coordinate axis, or if the control image is generated from an original image rotated at a known angle, the rotation angle can be directly obtained from the generation process. Then, a control image label set can be generated based on the label corresponding to each control image and the center point coordinates, border, and rotation angle of the control.

[0084] Step S730, obtaining a target detection model based on training of the control image label set.

[0085] Specifically, first, the control image label set is divided into a training set, a validation set, and a test set. Usually, the training set is used to train the model, the validation set is used to adjust the model parameters (such as hyperparameter tuning), and the test set is used to evaluate the final performance of the model. In addition, in order to improve the generalization ability of the model, the training set can be enhanced by data, such as rotation, scaling, cropping, flipping, etc. Then, according to specific requirements (such as real-time requirements, accuracy requirements, computing resource limitations, etc.), select a suitable target detection model. Common target detection models include models based on convolutional neural networks (CNNs), such as YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), Faster R-CNN, etc. Use a deep learning framework (such as TensorFlow, PyTorch, etc.) to load the preprocessed control images and corresponding label data. Define the model in the deep learning framework according to the selected target detection model architecture. This usually involves setting parameters such as the number of network layers, convolution kernel size, activation function, etc. : Set various parameters in the training process, such as learning rate, batch size, training rounds, etc. The training data is input into the model, the loss value is calculated through forward propagation, and then the model parameters are updated through back propagation. This process is repeated until the preset training rounds are reached or other stopping conditions are met.

[0086] For example, take the target detection model as the yolov5 model. First, because yolov5 uses a specific data format (usually COCO or VOC format, but yolov5 has its own data organization method). You need to convert the control image label set into a format that yolov5 can recognize. This usually involves creating a text file (such as a .txt file) containing the image path, category ID, and bounding box coordinates. Divide the dataset into a training set, a validation set, and an (optional) test set. Yolov5 uses configuration files (such as yolov5s.yaml, yolov5m.yaml, etc.) to define model architecture, training parameters, etc. These configuration files need to be modified according to your dataset and task requirements. In particular, you need to specify the number of categories, the path to the training / validation dataset, etc. Then, use the training script provided by yolov5 (such as train.py) to train the model. You need to specify the configuration file and dataset path as command line parameters. The training process will output training logs, including indicators such as loss value and accuracy. During the training process, yolov5 will automatically evaluate the performance of the model on the validation set. In some feasible embodiments, the test set can also be used after the training is completed to further verify the accuracy and robustness of the model, thereby obtaining the yolov5 model.

[0087] In this embodiment, by generating a control image label set based on the control image set and its labels, and training the target detection model accordingly, significant beneficial effects are brought about in terms of improving model accuracy, accelerating the training process, supporting complex interface detection, reducing labor costs, and improving testing efficiency.

[0088] Further, based on the above embodiments, please refer to Figure 8 In one of the exemplary embodiments provided in the present application, the specific implementation process of the above-mentioned automated testing method may further include step S810 and step S820, which are described in detail as follows:

[0089] Step S810, parsing the scenario to be tested to obtain key functions corresponding to the scenario to be tested;

[0090] Step S820: determining the corresponding control label based on the key function, and searching for a matching target control based on the control label.

[0091] Specifically, you can understand user behavior, input data, expected results, etc. by parsing the test scenario. Convert the scenario description into a business process diagram or a list of steps to clarify the operations and input and output of each step. Then find the corresponding user interface (UI) controls for each key function, which are the direct targets for performing test operations (such as clicks, inputs, selections, etc.). Analyze the mapping relationship between controls and functions: Based on the UI design of the application, analyze which controls are directly related to the extracted key functions. Determine control labels: Define unique labels or identifiers for these controls. These labels can be the ID, class name, text content, or other attributes of the control, depending on the type of control and the capabilities of the test framework. Write control search logic: Based on the control labels, write code or scripts to locate these controls in the UI. This usually involves using the API provided by the test framework to search for elements with specific attributes.

[0092] In this embodiment, by parsing the scenario to be tested, determining the control labels corresponding to key functions, and searching for matching target controls, significant beneficial effects are brought about in terms of improving the targetedness, degree of automation, comprehensiveness, depth of the test, reducing costs, and improving the flexibility and scalability of the test.

[0093] Further, based on the above embodiments, please refer to Fig. 9 In one of the exemplary embodiments provided in the present application, the specific implementation process of the above-mentioned automated testing method may further include steps S910 to S930, which are described in detail as follows:

[0094] Step S910, determining the expected result corresponding to the scenario to be tested;

[0095] Step S920, after the automatic control instruction is executed, the operation result corresponding to the target control is obtained;

[0096] Step S930: If the operation result matches the expected result, it is determined that the test of the target control passes.

[0097] Specifically, based on the scenario requirements, clearly define the results that should be observed after the test is executed. This may include changes in the UI interface, data updates, system response status, etc. Record the expected results clearly in the test plan so that they can be verified after the test is executed. After the script is executed, capture the operation results of the target control through appropriate means (such as API calls, UI element inspections, log analysis, etc.). This may involve obtaining the attribute values ​​of interface elements, checking data changes in the database, verifying records in the system log, etc. Use assertions or similar mechanisms to compare the actual operation results with the expected results set previously. This usually involves operations such as string comparison, numerical comparison, and status checking. Evaluate whether the test passes based on the comparison results. If the operation result completely matches the expected result, the test of the target control is considered to have passed; otherwise, the test is considered to have failed.

[0098] In this embodiment, by determining the expected results, comparing the operation results with the expected results to determine whether the test passes, significant beneficial effects are brought about in terms of improving the reliability and effectiveness of the test, supporting automated testing, enhancing the feedback and iteration capabilities of the test, and improving product quality and user satisfaction.

[0099] Fig.10It is a brief flow diagram of automated testing in an exemplary application scenario. In the application scenario shown in 10, the scene to be tested is parsed to obtain the key function corresponding to the scene to be tested; the corresponding control label is determined based on the key function, and the matching target control and the target state corresponding to the target control are found based on the control label, and the original state of the target control in the current visual interface is obtained, and the original state includes the original center point coordinates, original border and original rotation angle of the target control. The target center point coordinates, target border and target rotation angle corresponding to the target control are determined based on the scene to be tested; multiple operation values ​​are determined based on the target center point coordinates, target border and target rotation angle and the original center point coordinates, original border and original rotation angle; multiple operations corresponding to the drive-level operation are generated based on the multiple operation values, and automated operation instructions are generated based on the multiple operations. The state of the target control is adjusted based on the automated control instruction to obtain the adjusted visual interface, and the actual state corresponding to the target control in the adjusted visual interface is extracted; the expected result corresponding to the scene to be tested is determined; after the automated control instruction is executed, the operation result corresponding to the target control is obtained; if the operation result matches the expected result, it is determined that the test of the target control passes. For the detailed implementation process, please refer to the records in the aforementioned embodiments, and this will not be repeated here.

[0100] Fig.11 is a block diagram of a road condition refreshing device shown in an exemplary embodiment of the present application. The device can be applied to Figure 2 The implementation environment shown in the figure is specifically configured in the controller 220. The device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applied.

[0101] like Fig.11 As shown, the exemplary automated testing device includes: a determination module 1110, used to determine a target control and a target state corresponding to the target control based on a scenario to be tested; an acquisition module 1120, used to obtain an original state of the target control in a current visual interface, and determine an automated operation instruction for the target control based on the original state and the target state; an adjustment module 1130, used to adjust the state of the target control based on the automated control instruction, obtain an adjusted visual interface, and extract an actual state corresponding to the target control in the adjusted visual interface; and a judgment module 1140, used to determine that the test of the target control has passed if the actual state matches the target state.

[0102] According to one aspect of an embodiment of the present application, the above-mentioned acquisition module 1120 is also used to perform image detection on the visual interface based on the target detection model to obtain the original detection result; if the confidence of the original detection result reaches above a preset confidence threshold, the original state of the target control is determined based on the detection result, and the original state includes the original center point coordinates, the original border and the original rotation angle of the target control.

[0103] According to one aspect of an embodiment of the present application, the above-mentioned acquisition module 1120 is also used to determine the target center point coordinates, target border and target rotation angle corresponding to the target control based on the scene to be tested; determine multiple operation values ​​based on the target center point coordinates, target border and target rotation angle and the original center point coordinates, original border and original rotation angle; generate multiple operations corresponding to the driving-level operation based on the multiple operation values, and generate automatic operation instructions based on the multiple operations.

[0104] According to one aspect of an embodiment of the present application, the above-mentioned adjustment module 1130 is also used to perform image detection on the adjusted visual interface based on the target detection model to obtain an adjusted detection result; if the confidence of the adjusted detection result reaches above a preset confidence threshold, the actual state corresponding to the target control is determined based on the adjusted detection result.

[0105] According to one aspect of an embodiment of the present application, the above-mentioned automated testing device also includes a training module, which is used to obtain a control image set in a visualization interface, and to annotate the control based on the position of the control on the visualization interface and the function of the control to obtain a label corresponding to the control; to generate a control image label set based on the control image set and the label corresponding to the control, the control image label set including the center point coordinates, border and rotation angle corresponding to the control; and to obtain a target detection model based on training the control image label set.

[0106] According to one aspect of an embodiment of the present application, the above-mentioned determination module 1110 is also used to parse the scenario to be tested to obtain the key functions corresponding to the scenario to be tested; determine the corresponding control label based on the key functions, and search for a matching target control based on the control label.

[0107] According to one aspect of an embodiment of the present application, the above-mentioned judgment module 1140 is also used to determine the expected result corresponding to the scenario to be tested; after the execution of the automated control instruction, obtain the operation result corresponding to the target control; if the operation result matches the expected result, it is determined that the test of the target control has passed.

[0108] It should be noted that the automated testing device provided in the above embodiment and the automated testing method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the automated testing device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0109] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the electronic device implements the automated testing method provided in the above-mentioned embodiments.

[0110] Fig.12 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Fig.12 The computer system 1200 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0111] like Fig.12 As shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage part 1208 to the random access memory (RAM) 1203, such as executing the method in the above embodiment. In the RAM 1203, various programs and data required for system operation are also stored. The CPU 1201, the ROM 1202 and the RAM 1203 are connected to each other through the bus 1204. The input / output (I / O) interface 1205 is also connected to the bus 1204.

[0112] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed so that a computer program read therefrom is installed into the storage section 1208 as needed.

[0113] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1209, and / or installed from a removable medium 1211. When the computer program is executed by a central processing unit (CPU) 1201, various functions defined in the system of the present application are executed.

[0114] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0115] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0116] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0117] Another aspect of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned automated testing method is implemented. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0118] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the automated testing method provided in each of the above embodiments.

[0119] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

Claims

1. An automated testing method, characterized in that: include: Determine a target control and a target state corresponding to the target control based on the scenario to be tested; Acquire the original state of the target control in the current visual interface, and determine the automated operation instruction of the target control based on the original state and the target state; Adjusting the state of the target control based on the automation control instruction to obtain an adjusted visual interface, and extracting the actual state corresponding to the target control in the adjusted visual interface; If the actual state matches the target state, it is determined that the test of the target control passes.

2. The method according to claim 1, characterized in that The method further comprises: Performing image detection on the visualization interface based on the target detection model to obtain an original detection result; If the confidence of the original detection result reaches above a preset confidence threshold, the original state of the target control is determined based on the detection result, and the original state includes the original center point coordinates, the original border and the original rotation angle of the target control.

3. The method according to claim 2, characterized in that The method further comprises: Determine the target center point coordinates, target border, and target rotation angle corresponding to the target control based on the scene to be tested; Determine a plurality of operation values ​​based on the target center point coordinates, the target frame, and the target rotation angle and the original center point coordinates, the original frame, and the original rotation angle; A plurality of operations corresponding to the driver-level operation are generated based on the plurality of operation values, and the automation operation instruction is generated based on the plurality of operations.

4. The method according to claim 2, characterized in that The method further comprises: Performing image detection on the adjusted visualization interface based on the target detection model to obtain an adjusted detection result; If the confidence of the adjusted detection result reaches above a preset confidence threshold, the actual state corresponding to the target control is determined based on the adjusted detection result.

5. The method according to claim 2, characterized in that The method further comprises: Acquire a set of control images in the visualization interface, and annotate the control images based on the position of the control on the visualization interface and the function of the control to obtain a label corresponding to the control; Generate a control image label set based on the control image set and the label corresponding to the control, wherein the control image label set includes the center point coordinates, border and rotation angle corresponding to the control; The target detection model is obtained by training based on the control image label set.

6. The method according to claim 5, characterized in that The method further comprises: Analyze the scenario to be tested to obtain key functions corresponding to the scenario to be tested; A corresponding control label is determined based on the key function, so as to search for a matching target control based on the control label.

7. The method according to claim 1, characterized in that The method further comprises: Determine the expected result corresponding to the scenario to be tested; After the automation control instruction is executed, obtaining the operation result corresponding to the target control; If the operation result matches the expected result, it is determined that the test of the target control passes.

8. An automated testing device, characterized in that: The device comprises: A determination module, used to determine a target control and a target state corresponding to the target control based on a scenario to be tested; An acquisition module, used to acquire the original state of the target control in the current visual interface, and determine the automatic operation instruction of the target control based on the original state and the target state; An adjustment module, configured to adjust the state of the target control based on the automation control instruction, obtain an adjusted visual interface, and extract an actual state corresponding to the target control in the adjusted visual interface; A judgment module is used to determine whether the test of the target control passes if the actual state matches the target state.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the automated testing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the automated testing method according to any one of claims 1 to 7.

Images