Test case execution method and device, electronic equipment and storage medium

Abstract

The invention discloses a test case execution method and device, electronic equipment and a storage medium, and relates to the technical field of vehicle testing, the method comprises the steps that a target test case is acquired, and the target test case is used for representing that a first execution result on a first vehicle-mounted device is a test case with execution failure; whether the target test case meets a repeated execution condition or not is determined, and the repeated execution condition is a condition for excluding script errors existing in the target case; if yes, determining a second in-vehicle infotainment device from the in-vehicle infotainment device resource pool, and allocating the target test case to the second in-vehicle infotainment device to obtain a second execution result of the target test case on the second in-vehicle infotainment device; if the second execution result is execution success, determining that the reason of execution failure of the first vehicle machine is that the test environment of the first vehicle machine has defects; and if the second execution result is execution failure, determining that product design of the first in-vehicle infotainment device and the second in-vehicle infotainment device has defects. According to the invention, the possibility of false report of the test case is reduced, and the accuracy of the test result is improved.

Description

Technical Field

[0001] This application relates to the field of vehicle testing technology, and in particular to a method, apparatus, electronic device, and storage medium for executing test cases. Background Technology

[0002] Currently, in the field of automated testing of in-vehicle infotainment systems, test cases are typically used to test various software programs on the system, and test reports are generated based on the test results.

[0003] However, traditional vehicle infotainment testing methods, even after a test fails in the testing environment, will still use the same test cases to execute the test again in the same vehicle infotainment testing environment. Since the vehicle infotainment environment before and after the test is the same vehicle infotainment system, the traditional test case execution method cannot distinguish the real reason for the test failure, and cannot avoid the problem of inaccurate test case results caused by the abnormality of a single vehicle infotainment system. This increases the possibility of false alarms in test cases, reduces the accuracy of test results and the testing stability of the vehicle infotainment system itself. Summary of the Invention

[0004] This application provides a method, apparatus, electronic device, and storage medium for executing test cases. The embodiments provided by this application solve the problems in the prior art that cannot distinguish the true cause of test failure and cannot avoid inaccurate test case results caused by the abnormality of a single vehicle system, which increases the possibility of false alarms in test cases and reduces the accuracy of test results and the test stability of the vehicle system itself. This application can distinguish the true cause of the failure of the target test case execution, reduce the possibility of false alarms in test cases, and improve the accuracy of test results and the test stability of the vehicle system itself.

[0005] In a first aspect, this application provides a method for executing test cases, the method comprising: Obtain target test cases, wherein the target test cases are used to characterize test cases whose first execution result on the first vehicle-mounted unit is execution failure; Determine whether the target test case meets the condition for repeated execution, wherein the condition for repeated execution is the condition to exclude the existence of script errors in the target test case; If the conditions are met, a second vehicle unit is determined from the vehicle unit resource pool, and the target test case is assigned to the second vehicle unit to obtain a second execution result of the target test case on the second vehicle unit, wherein the second vehicle unit is different from the first vehicle unit; If the second execution result is successful, then the reason for the failure of the first vehicle system is determined to be a defect in the test environment of the first vehicle system. If the second execution result is execution failure, it is determined that there is a product design defect in both the first vehicle system and the second vehicle system.

[0006] In one feasible implementation, determining whether the target test case meets the repeated execution condition includes: Determine the failure type of the first execution result; If the failure type belongs to a preset failure type set, then the target test case is determined to meet the condition for repeated execution; wherein, the preset failure type set includes a set of failure types caused by vehicle system anomalies.

[0007] In one feasible implementation, determining the second vehicle unit from the vehicle unit resource pool includes: Obtain the load status and historical health of each candidate vehicle in the vehicle resource pool; The second vehicle unit is determined from the candidate vehicle units based on their load status and historical health status.

[0008] In one feasible implementation, determining the second vehicle unit from the candidate vehicle units based on their load status and historical health status includes: Based on the load status of each candidate vehicle unit, at least one candidate vehicle unit with an idle load status is selected from the vehicle unit resource pool. Based on the historical health data of each of the candidate vehicle systems, the health level of each candidate vehicle system is determined; The health levels of each candidate vehicle unit are sorted, and the second vehicle unit is selected from the candidate vehicle units whose health levels are higher than a preset health level threshold.

[0009] In one feasible implementation, the method further includes: If the second execution result is an execution failure, then the second execution result is used as the updated first execution result, and the process returns to determine whether the target test case meets the repeated execution condition, until the number of times the target test case is executed reaches the preset number.

[0010] In one feasible implementation, the method further includes: Obtain the frequency of execution failures of the first execution result within a preset time period; When the frequency is not less than the preset rerun test threshold, the first vehicle unit is added to the vehicle unit resource blacklist, and the allocation of new target test cases to the first vehicle unit is stopped.

[0011] In one feasible implementation, the method further includes: Based on the first execution result, the second execution result, the first identification information of the first vehicle system, and the second identification information of the second vehicle system, a target test report is generated.

[0012] In a second aspect, this application provides an execution apparatus for the test cases, comprising: The first acquisition module is used to acquire target test cases, wherein the target test cases are used to characterize test cases whose first execution result on the first vehicle-mounted unit is execution failure; The first determining module is used to determine whether the target test case meets the repeated execution condition, wherein the repeated execution condition is the condition that excludes the existence of script errors in the target test case; The module is configured to determine a second vehicle unit from the vehicle unit resource pool if the conditions are met, and to assign the target test case to the second vehicle unit, thereby obtaining a second execution result of the target test case on the second vehicle unit, wherein the second vehicle unit is different from the first vehicle unit; The second determining module is used to determine that if the second execution result is successful, the reason for the failure of the first vehicle system is that there is a defect in the test environment of the first vehicle system; The third determining module is used to determine that there is a product design defect in the first vehicle system and the second vehicle system if the second execution result is an execution failure.

[0013] In a third aspect, this application provides an electronic device, including a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus, and the machine-readable instructions are executed by the processor to perform the steps of the test case execution method described above.

[0014] In a fourth aspect, this application provides a computer-readable storage medium storing a computer program that, when run by a processor, executes the steps of the test case execution method described above.

[0015] Compared with the prior art, the test case execution method, apparatus, electronic device, and storage medium provided in this application embodiment obtain a first execution result of the target test case on a first vehicle infotainment system. If the first execution result is a failure, the embodiment determines whether the target test case meets the condition for repeated execution. If the target test case meets the condition for repeated execution, a second vehicle infotainment system is determined from the vehicle infotainment system resource pool, and the target test case is assigned to the second vehicle infotainment system to obtain a second execution result of the target test case on the second vehicle infotainment system. If the second execution result is a failure, the second execution result is used as a new first execution result, and the process returns to the step of determining whether the target test case meets the condition for repeated execution when the first execution result is a failure, until the number of executions of the target test case reaches a preset number. Then, based on the first execution result and the second execution result, the target judgment result of the target test case is determined. This application reduces the possibility of false positives in test cases and improves the accuracy of test results.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a test case execution method provided in an embodiment of this application is shown. Figure 2 The flowchart illustrating the stopping of a target test case in a test case execution method provided in an embodiment of this application is shown. Figure 3 This paper shows a structural block diagram of a test case execution device provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown.

[0018] Figure 3 and Figure 4 The correspondence between the figure labels and figure titles in the accompanying drawings is as follows: 300 Test case execution device; 310 First acquisition module; 320 First determination module; 330 Obtaining module; 340 Second determination module; 350 Third determination module; 360 Generation module; 370 Second acquisition module; 380 Stopping module; 390 Update module; 400 Electronic device; 410 Processor; 420 Memory; 430 Bus. Detailed Implementation

[0019] To better understand the technical solutions provided in the embodiments of this specification, the technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this specification, rather than limitations on the technical solutions of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.

[0020] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The term "two or more" includes two or more cases.

[0021] First, the applicable application scenarios of this application are introduced. The embodiments provided in this application are applicable to the field of vehicle testing technology, and in particular relate to a test case execution method, device, electronic device and storage medium.

[0022] Currently, traditional vehicle infotainment testing methods, even after a test fails in the testing environment, still use the same test cases to execute the test again in the same vehicle infotainment testing environment. Since the vehicle infotainment environment before and after the test is the same vehicle infotainment system, the traditional test case execution method cannot distinguish the real reason for the test failure, and cannot avoid the problem of inaccurate test case results caused by the abnormality of a single vehicle infotainment system. This increases the possibility of false alarms in test cases, reduces the accuracy of test results and the testing stability of the vehicle infotainment system itself.

[0023] Based on this, the embodiments of this application provide a testing method, device, electronic device, and storage medium for vehicle-mounted systems under multiple environments. The embodiments provided by this application solve the problems in the prior art that cannot distinguish the true cause of test failure and cannot avoid inaccurate test case results caused by the abnormality of a single vehicle-mounted system itself, which increases the possibility of false alarms in test cases and reduces the accuracy of test results and the testing stability of the vehicle-mounted system itself. This application can distinguish the true cause of the failure of the target test case execution, reduce the possibility of false alarms in test cases, and improve the accuracy of test results and the testing stability of the vehicle-mounted system itself.

[0024] In one exemplary embodiment, Figure 1 A flowchart illustrating a test case execution method provided in an embodiment of this application is shown. The testing method for in-vehicle infotainment systems under multiple environments includes the following steps: S101. Obtain the target test case, wherein the target test case is used to characterize the test case whose first execution result on the first vehicle-mounted machine is execution failure.

[0025] In this step, in the embodiments provided in this application, test tasks automatically triggered by users and / or vehicle systems are first received, and then the test scheduling center component on the vehicle system is used to manage the test tasks and assign them to the test cases corresponding to the test tasks. The test is then executed on the first vehicle system, and finally a first execution result is generated. Then, the test cases whose first execution result on the first vehicle system is a failure are marked as target test cases.

[0026] In the embodiments provided in this application, if the execution of the first execution result is determined to be successful, the test of the first vehicle-mounted machine is directly determined to be successful, and the test process of the test case is determined to be over.

[0027] In the above, the first vehicle-mounted unit is the vehicle-mounted unit that executes the target test case for the first time.

[0028] It is understood that all vehicle-mounted units, including the first and second vehicle-mounted units, in the embodiments provided in this application are connected to the test dispatch center component via a network (such as ADB Fire Wi-Fi).

[0029] It should be noted that the target test cases in the embodiments provided in this application may be specifically determined by user instructions, script generation, and continuous integration (CI) triggering.

[0030] S102. Determine whether the target test case meets the repeated execution condition, where the repeated execution condition is the condition to exclude the existence of script errors in the target test case.

[0031] In this step, the embodiment provided in this application, after determining that the first execution result on the first vehicle-mounted machine is an execution failure, receives the target test case and repeatedly executes the target test case. Instead of rerunning all failed target test cases, such as execution failures caused by script syntax errors, rerunning is meaningless. Therefore, it is necessary to analyze the target test cases to determine whether the target test cases meet the conditions for repeated execution.

[0032] It is understood that the repeated execution conditions in the embodiments provided in this application can be customized and used according to different application scenarios and usage conditions. The repeated execution conditions in the embodiments provided in this application are specific but not limited to: only when the failure of the first execution result is caused by the abnormality of a single vehicle unit itself, the same target test case can be tested on other vehicle units.

[0033] It should be noted that the embodiments provided in this application can be specifically automated, and can monitor, capture, summarize and report the execution status and results of all target test cases on various vehicle-mounted systems in real time.

[0034] The first execution result in the embodiments provided in this application includes execution success, execution failure, execution error, and execution timeout.

[0035] S103. If satisfied, the second vehicle unit is determined from the vehicle unit resource pool, and the target test case is assigned to the second vehicle unit to obtain the second execution result of the target test case on the second vehicle unit, wherein the second vehicle unit is different from the first vehicle unit.

[0036] In this step, after analyzing and determining that the target test case meets the repeated execution conditions, the embodiment provided in this application begins to determine a second vehicle unit that can execute the target test case from the pre-established vehicle unit resource pool, and determines the second execution result of the target test case on the second vehicle unit.

[0037] It is understood that the second execution result in the embodiments provided in this application may be the same as or different from the first execution result described above; the second vehicle unit in the embodiments provided in this application is used to characterize other healthy vehicle units in the vehicle unit resource pool, and the second vehicle unit is a vehicle unit that is not executing the target test case for the first time.

[0038] It should be noted that in the embodiments provided in this application, whether it is the first vehicle-mounted unit, the second vehicle-mounted unit, or other types of vehicle-mounted units in the vehicle-mounted unit resource pool, each vehicle-mounted unit has a unique identification information, and each vehicle-mounted unit can report its own load status in real time through the unique identification information, such as idle status, busy status, offline status, and abnormal status.

[0039] S104. If the second execution result is successful, then the reason for the failure of the first vehicle system is determined to be a defect in the test environment of the first vehicle system.

[0040] In the embodiments provided in this application, after determining that the second execution result is successful, it is explained that the reason why the first execution result of the first vehicle unit failed is due to the misjudgment of the test environment of the first vehicle unit. Therefore, it is determined that the test environment of the first vehicle unit has defects.

[0041] In this step, the embodiment provided in this application determines that the target test was successfully executed after the second execution result is determined to be successful. Based on the successful second execution result and the unsuccessful first execution result, a target test report is generated or updated. Then, all information during the execution process, including the target test cases, the established test tasks, the execution record of the target test, and the record of repeated execution (i.e., rerun) of the test, is stored to facilitate subsequent traceability by the operators.

[0042] It is understood that, after determining that the second execution result is execution failure, the embodiments provided in this application will repeatedly execute the step of determining whether the target test case meets the repeated execution condition when the first execution result is execution failure, and then continue to run the target test case again and assign it to other vehicle units until the number of executions of the target test case reaches the preset number, and the second vehicle unit reassigned can be different from the previously assigned second vehicle unit.

[0043] It should be noted that the preset number of times in the embodiments provided in this application can be customized and used according to different application scenarios and usage conditions. The preset number of times in the embodiments provided in this application can be set to 1-2 times to prevent infinite reruns.

[0044] If the embodiment provided in this application fails to execute the target test case successfully after the target test case has been executed a preset number of times, the testing of the target test case will be suspended.

[0045] S105. If the second execution result is execution failure, it is determined that there is a product design defect in the first vehicle unit and the second vehicle unit.

[0046] In this step, after determining that the second execution result is an execution failure, it is explained that the reason why the first execution result of the first vehicle system failed to succeed and the reason why the second execution result of the second vehicle system failed to succeed are the same, both due to defects in their own product design or the vehicle system itself.

[0047] Compared with the prior art, the test case execution method provided in this application obtains the first execution result of the target test case on the first vehicle-mounted system. If the first execution result is a failure, it determines whether the target test case meets the condition for repeated execution. If the target test case meets the condition for repeated execution, it determines the second vehicle-mounted system from the vehicle-mounted system resource pool and assigns the target test case to the second vehicle-mounted system to obtain the second execution result of the target test case on the second vehicle-mounted system. If the second execution result is a failure, it takes the second execution result as the new first execution result and returns to the step of determining whether the target test case meets the condition for repeated execution if the first execution result is a failure, until the number of executions of the target test case reaches a preset number. Then, based on the first execution result and the second execution result, it determines the target judgment result of the target test case. This application reduces the possibility of false positives in test cases and improves the accuracy of test results.

[0048] As an optional implementation method in this embodiment, determining whether the target test case meets the repeated execution condition includes: Determine the failure type of the first execution result; if the failure type belongs to the preset failure type set, then determine that the target test case meets the condition for repeated execution; wherein, the preset failure type set includes the set of failure types caused by vehicle system anomalies.

[0049] It is understood that, in the embodiments provided in this application, when the first execution result is determined to be an execution failure, the failure type of the first execution result will be further determined, and based on the failure type, it will be determined whether the target test case meets the condition for repeated execution. If it is determined that the failure type does not belong to the preset failure type set, the target test case will not be executed repeatedly. That is, not all failed first execution results need to be rerun. In the embodiments provided in this application, only the first execution results whose failure type belongs to the preset failure type set will be executed repeatedly.

[0050] It should be noted that the preset failure type set provided in the embodiments of this application can be customized and used according to different application scenarios and usage conditions. The preset failure type set provided in the embodiments of this application is specifically, but not limited to, situations such as UI element not found, application crash, and assertion error.

[0051] For example, if the failure type of the first execution result does not belong to the preset failure type set, such as when the failure type characteristic is a script syntax error, it is determined that the test case will not be rerun and the target judgment result of the target test case will be determined as failure.

[0052] In this application, comparing the failure type with the preset failure type in the preset failure type set can avoid meaningless reruns of the target test case, thereby reducing the misjudgment of the target judgment result caused by the abnormality of a single vehicle machine, and thus improving the stability and accuracy of the automated test target test case execution test.

[0053] As an optional implementation method in this embodiment, determining the second vehicle unit from the vehicle unit resource pool includes: obtaining the load status and historical health of each candidate vehicle unit in the vehicle unit resource pool; and determining the second vehicle unit from each candidate vehicle unit based on the load status and historical health of each candidate vehicle unit.

[0054] In the above-described embodiments, the present application will select at least one candidate vehicle unit with an idle load status from the vehicle unit resource pool based on the load status of each candidate vehicle unit; determine the health level of each candidate vehicle unit based on the historical health data of each candidate vehicle unit; sort the health levels of each candidate vehicle unit; and select a second vehicle unit from the candidate vehicle units whose health level is higher than a preset health level threshold.

[0055] In the above-described embodiments, after determining that the target test cases meet the repeated execution conditions, the first step is to determine the load status and historical health of each candidate vehicle in the vehicle resource pool. Then, at least one candidate vehicle with an idle load status is selected from the vehicle resource pool, i.e., the least one candidate vehicle with the most idle load status. If there is only one candidate vehicle, it is directly selected as the second vehicle. If there are two or more candidate vehicles with the most idle load status, the candidate vehicle with the highest health level is selected as the second vehicle.

[0056] Here, if the second vehicle unit that performs repeated tests in the embodiments provided in this application has specific hardware and system version requirements, then the above-mentioned feature requirements should be met first, and then the system should be selected in order of load status and health level.

[0057] As an optional implementation method in this embodiment, the method further includes: If the second execution result is an execution failure, the second execution result will be used as the updated first execution result, and the process will return to determine whether the target test case meets the repeated execution condition, until the number of times the target test case is executed reaches the preset number.

[0058] In this step, the embodiment provided in this application determines that the target test was successfully executed after the second execution result is determined to be successful. Based on the successful second execution result and the unsuccessful first execution result, a target test report is generated or updated. Then, all information during the execution process, including the target test cases, the established test tasks, the execution record of the target test, and the record of repeated execution (i.e., rerun) of the test, is stored to facilitate subsequent traceability by the operators.

[0059] It is understood that, after determining that the second execution result is execution failure, the embodiments provided in this application will repeatedly determine whether the target test case meets the repeated execution condition, and then continue to run the target test case again and assign it to other second vehicle units until the number of executions of the target test case reaches the preset number, and the reassigned second vehicle unit can be different from the previously assigned second vehicle unit, that is, the number of second vehicle units is at least one.

[0060] It should be noted that the preset number of times in the embodiments provided in this application can be customized and used according to different application scenarios and usage conditions. The preset number of times in the embodiments provided in this application can be set to 1-2 times to prevent infinite reruns.

[0061] If the embodiment provided in this application fails to execute the target test case successfully after the target test case has been executed a preset number of times, the testing of the target test case will be suspended.

[0062] It should be noted that if the second execution result in the embodiments provided in this application is different from the first execution result, the target judgment result of the target test case is determined to be successful, and a first judgment note information containing the first vehicle system error is generated. The first judgment note information may specifically indicate that there is a defect in the test environment. If the second execution result is the same as the first execution result, the target judgment result of the target test case is determined to be successful, and a second judgment note information containing the test product defect prompt is generated. The second judgment note information may specifically indicate that there is a defect in the product design of the first vehicle system and the second vehicle system.

[0063] It should be noted that, in the embodiments provided in this application, when the second execution result is different from the first execution result, it indicates that the target test case was ultimately executed successfully, and further indicates that the first execution failure of the target test case was caused by an abnormal single-device environment, and a first judgment note information containing the first vehicle system abnormality is generated; when the second execution result is the same as the first execution result, the target judgment result of the target test case is determined to be that the test failed, and a second judgment note information containing the test product defect prompt is generated, so as to effectively distinguish between real product defects and test environment instability, and greatly reduce false alarms.

[0064] Based on the above embodiments, the method further includes a method for stopping the allocation of new target test cases to the first vehicle-mounted system. Figure 2 This paper illustrates a flowchart of a test case execution method according to an embodiment of the present application, which involves stopping a target test case. The method further includes the following steps: S201. Obtain the frequency of execution failures of the first execution result within a preset time period.

[0065] It should be noted that the preset time period in the embodiments provided in this application can be customized and used according to different application scenarios and usage conditions.

[0066] S202. When the frequency is not less than the preset rerun test threshold, add the first vehicle unit to the vehicle unit resource blacklist and stop assigning new target test cases to the first vehicle unit.

[0067] It should be noted that in the embodiments provided in this application, when the frequency of execution failure within a preset time period is greater than or equal to the preset rerun test threshold, it is determined that the first vehicle unit will frequently cause the above target test cases to fail in a short period of time. It will automatically add the vehicle unit to the vehicle unit resource blacklist, suspend automatic rerun and issue an alarm, and notify the operators to check, so as to prevent large-scale problems from occurring in the entire test environment (such as servers or networks).

[0068] Here, the embodiments provided in this application are integrated with Continuous Integration, CI / ContinuousDeliveryCD, so that the pipeline is blocked only if the target test case still fails after rerun.

[0069] As an optional implementation in this embodiment, after determining the target judgment result of the target test case based on the first execution result and the second execution result, the method further includes: generating a target test report based on the first execution result, the second execution result, the first identification information of the first vehicle unit and the second identification information of the second vehicle unit.

[0070] It should be noted that the embodiments provided in this application will associate and bind the first execution result, the second execution result, the first identification information of the first vehicle unit, and the second identification information of the second vehicle unit to generate a target test report.

[0071] This application will assign the target test case to another healthy and idle second vehicle unit after the target test case fails to execute on the first vehicle unit. It will not directly judge the target test case as a failure, and then make the final target judgment result based on the charging result.

[0072] The following examples illustrate the specific execution methods for different types of test cases: Example 1: The first vehicle-mounted system is represented by A, and the second vehicle-mounted system is represented by B.

[0073] Sub-step 1. Execute the target test case on A for the first time.

[0074] Sub-step 2. Receive the first execution result of the target test case failing to execute on A.

[0075] Sub-step 3. Retry executing the above target test cases on B.

[0076] Sub-step 4. Receive the target judgment result of the successful execution of the target test case on B, and generate the first judgment note information containing the first vehicle machine anomaly, so as to help the operators quickly understand the reason for the failure of the first execution result.

[0077] Example 2: Suppose the target test case is set as: login verification test, test_login.

[0078] Sub-step 1. Perform the verification login test on A and receive the result of the execution failure. The specific reason for the execution failure is: the network module of A experiences a momentary failure.

[0079] Sub-step 2. After determining that the failure type of the login verification test belongs to the preset failure type set, assign the login verification test to B for execution and receive the result of successful execution.

[0080] Sub-step 3. Finally, determine the target test case result of the above "verification login test" as successful execution, and complete the target test report of "failed on A on the first attempt, and succeeded on B on the retry" to remind subsequent testers or operators that they do not need to pay special attention to the reasons for the above failure.

[0081] Example 3: Suppose the target test case is set as: login verification test, test_login.

[0082] Sub-step 1. Perform a login verification test on A and receive the result of the execution failure. The specific reason for the failure is: the login button is unresponsive.

[0083] Sub-step 2. After determining that the failure type of the login verification test belongs to the preset failure type set, assign the login verification test to B for execution, and receive the execution failure result again.

[0084] Sub-step 3. Finally determine that the target test case of the above "Verification Login Test" has failed to execute, and complete the target test report "Failed on both A and B" to remind subsequent testers or operators that the reason for the failure is a real product bug and that testers need to investigate it immediately.

[0085] The test case execution method provided in this application can greatly save testers' time in troubleshooting environmental issues, thereby increasing work efficiency and reducing testing costs.

[0086] Please see Figure 3 , Figure 3 This diagram illustrates a structural block diagram of a test case execution device provided in an embodiment of this application. Figure 3 As shown, the test case execution device 300 includes: The first acquisition module 310 is used to acquire target test cases, wherein the target test cases are used to characterize test cases whose first execution result on the first vehicle-mounted machine is execution failure.

[0087] The first determining module 320 is used to determine whether the target test case meets the repeated execution condition, wherein the repeated execution condition is the condition to exclude the existence of script errors in the target test case.

[0088] Module 330 is used to determine the second vehicle unit from the vehicle unit resource pool if the conditions are met, and to assign the target test case to the second vehicle unit, thereby obtaining the second execution result of the target test case on the second vehicle unit, wherein the second vehicle unit is different from the first vehicle unit.

[0089] The second determining module 340 is used to determine, if the second execution result is successful, that the reason for the failure of the first vehicle system is that there is a defect in the test environment of the first vehicle system.

[0090] The third determining module 350 is used to determine that there is a product design defect in the first vehicle unit and the second vehicle unit if the second execution result is an execution failure.

[0091] The generation module 360 ​​is used to generate a target test report based on the first execution result, the second execution result, the first identification information of the first vehicle unit, and the second identification information of the second vehicle unit.

[0092] The second acquisition module 370 is used to acquire the frequency of failure of the first execution result within a preset time period.

[0093] The stop module 380 is used to add the first vehicle unit to the vehicle unit resource blacklist and stop allocating new target test cases to the first vehicle unit when the frequency is not less than the preset rerun test threshold.

[0094] The update module 390 is used to take the second execution result as the updated first execution result if the second execution result is an execution failure, and return to the execution to determine whether the target test case meets the repeated execution condition, until the number of executions of the target test case reaches the preset number.

[0095] For example, the first determining module 320 is specifically used for: Determine the failure type of the first execution result.

[0096] If the failure type belongs to the preset failure type set, then the target test case is determined to meet the condition for repeated execution; the preset failure type set includes the set of failure types caused by vehicle system anomalies.

[0097] For example, module 330 is obtained, specifically for: Obtain the load status and historical health of each candidate vehicle in the vehicle resource pool.

[0098] Based on the load status and historical health of each candidate vehicle unit, the second vehicle unit is determined from among the candidate vehicle units.

[0099] For example, based on the load status and historical health of each candidate vehicle-mounted unit, a second vehicle-mounted unit is determined from the candidate units, including: Based on the load status of each candidate vehicle-mounted unit, at least one candidate vehicle-mounted unit with an idle load status is selected from the vehicle-mounted unit resource pool.

[0100] Based on the historical health data of each candidate vehicle infotainment system, the health level of each candidate vehicle infotainment system is determined.

[0101] The health levels of each candidate vehicle unit are sorted, and the second vehicle unit is selected from those with a health level higher than the preset health level threshold.

[0102] Compared with the prior art, the test case execution device 300 provided in this application obtains the first execution result of the target test case on the first vehicle infotainment system. If the first execution result is an execution failure, it determines whether the target test case meets the repeated execution condition. If the target test case meets the repeated execution condition, it determines the second vehicle infotainment system from the vehicle infotainment system resource pool and assigns the target test case to the second vehicle infotainment system to obtain the second execution result of the target test case on the second vehicle infotainment system. If the second execution result is an execution failure, it takes the second execution result as the new first execution result and returns to the step of determining whether the target test case meets the repeated execution condition if the first execution result is an execution failure, until the number of executions of the target test case reaches a preset number. Then, based on the first execution result and the second execution result, it determines the target judgment result of the target test case. This application reduces the possibility of false positives in test cases and improves the accuracy of test results.

[0103] In this application, comparing the failure type with the preset failure type in the preset failure type set can avoid meaningless reruns of the target test case, thereby reducing the misjudgment of the target judgment result caused by the abnormality of a single vehicle machine, and thus improving the stability and accuracy of the automated test target test case execution test.

[0104] The test case device 300 provided in this application can greatly save testers' time in troubleshooting environmental problems, thereby increasing work efficiency and reducing testing costs.

[0105] Please see Figure 4 , Figure 4 This application provides a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0106] Memory 420 stores machine-readable instructions executable by processor 410. When electronic device 400 is running, processor 410 and memory 420 communicate via bus 430. When the machine-readable instructions are executed by processor 410, they can perform the operations described above. Figures 1 to 2 The steps of the method for determining vehicle faults in the method embodiment shown in the method embodiment for executing test cases are described in detail in the method embodiment, and will not be repeated here.

[0107] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figures 1 to 2The steps of the method for determining vehicle faults in the method embodiment shown in the method embodiment for executing test cases are described in detail in the method embodiment, and will not be repeated here.

[0108] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0109] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0110] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.

[0111] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0112] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0114] This application also provides a computer program product, which includes computer software instructions that, when executed on a processing device, cause the processing device to execute a method for executing test cases.

[0115] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0116] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0120] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0121] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0122] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this specification.

[0123] Obviously, those skilled in the art can make various modifications and variations to this specification without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims and their equivalents, this specification is also intended to include such modifications and variations.

Claims

1. A method for executing test cases, characterized in that, The method includes: Obtain target test cases, wherein the target test cases are used to characterize test cases whose first execution result on the first vehicle-mounted unit is execution failure; Determine whether the target test case meets the condition for repeated execution, wherein the condition for repeated execution is the condition to exclude the existence of script errors in the target test case; If the conditions are met, a second vehicle unit is determined from the vehicle unit resource pool, and the target test case is assigned to the second vehicle unit to obtain a second execution result of the target test case on the second vehicle unit, wherein the second vehicle unit is different from the first vehicle unit; If the second execution result is successful, then the reason for the failure of the first vehicle system is determined to be a defect in the test environment of the first vehicle system. If the second execution result is execution failure, it is determined that there is a product design defect in both the first vehicle system and the second vehicle system.

2. The method for executing test cases according to claim 1, characterized in that, Determining whether the target test case meets the condition for repeated execution includes: Determine the failure type of the first execution result; If the failure type belongs to a preset failure type set, then the target test case is determined to meet the condition for repeated execution; wherein, the preset failure type set includes a set of failure types caused by vehicle system anomalies.

3. The method for executing test cases according to claim 1, characterized in that, The step of determining the second vehicle-mounted unit from the vehicle-mounted unit resource pool includes: Obtain the load status and historical health of each candidate vehicle in the vehicle resource pool; The second vehicle unit is determined from the candidate vehicle units based on their load status and historical health status.

4. The method for executing test cases according to claim 3, characterized in that, The step of determining the second vehicle unit from the candidate vehicle units based on their load status and historical health status includes: Based on the load status of each candidate vehicle unit, at least one candidate vehicle unit with an idle load status is selected from the vehicle unit resource pool. Based on the historical health status of each of the candidate vehicle systems, the health status level of each candidate vehicle system is determined. The health levels of each candidate vehicle unit are sorted, and a second vehicle unit is selected from the candidate vehicle units whose health levels are higher than a preset health level.

5. The method for executing test cases according to claim 1, characterized in that, The method further includes: If the second execution result is an execution failure, then the second execution result is used as the updated first execution result, and the process returns to determine whether the target test case meets the repeated execution condition, until the number of times the target test case is executed reaches the preset number.

6. The method for executing test cases according to claim 1, characterized in that, The method further includes: Obtain the frequency of execution failures of the first execution result within a preset time period; When the frequency is not less than the preset rerun test threshold, the first vehicle unit is added to the vehicle unit resource blacklist, and the allocation of new target test cases to the first vehicle unit is stopped.

7. The method for executing test cases according to claim 1, characterized in that, The method further includes: Based on the first execution result, the second execution result, the first identification information of the first vehicle system, and the second identification information of the second vehicle system, a target test report is generated.

8. A test case execution device, characterized in that, The execution device for the test cases includes: The first acquisition module is used to acquire target test cases, wherein the target test cases are used to characterize test cases whose first execution result on the first vehicle-mounted unit is execution failure; The first determining module is used to determine whether the target test case meets the repeated execution condition, wherein the repeated execution condition is the condition that excludes the existence of script errors in the target test case; The module is configured to determine a second vehicle unit from the vehicle unit resource pool if the conditions are met, and to assign the target test case to the second vehicle unit, thereby obtaining a second execution result of the target test case on the second vehicle unit, wherein the second vehicle unit is different from the first vehicle unit; The second determining module is used to determine that if the second execution result is successful, the reason for the failure of the first vehicle system is that there is a defect in the test environment of the first vehicle system; The third determining module is used to determine that there is a product design defect in the first vehicle system and the second vehicle system if the second execution result is an execution failure.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus, and the machine-readable instructions are executed by the processor to perform the steps of the test case execution method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the test case execution method as described in any one of claims 1-7.

Images