Test case collection method, device, electronic device and target vehicle

Abstract

The present invention relates to the field of software testing technology, and specifically to a test case collection method, device, electronic device and target vehicle. Continuously monitor the functional test behavior of the target software; the target software is an in-vehicle voice interaction software; the functional test behavior is used to test the interaction between the target software and the target vehicle; if the functional test behavior of the target software is monitored, then capture the various dynamic logs in the process of executing the functional test behavior in real time; identify the dynamic logs, and generate test cases corresponding to the functional test behavior; store the test cases in a preset test case database to achieve the reuse of the test cases. The efficiency of generating test cases is improved, and the storage of test cases is realized, and then the test cases can be reused through the test cases stored in the preset test case database, thereby improving the testing efficiency of the in-vehicle voice interaction software and reducing labor costs.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a test case collection method, a device, an electronic device and a target vehicle. Background Art

[0002] Faced with the current situation of diversified car models and rapid iteration of in-vehicle voice interaction assistant versions, traditional functional testing methods for voice interaction assistants have shown their limitations. The main manifestations are low testing efficiency and high labor costs. Each iteration means starting from scratch or modifying the existing test process in large quantities. During the testing process, it seems difficult to cope with frequent changes, which not only consumes precious resources but also restricts the response speed of the test team.

[0003] Therefore, how to achieve the reuse of test cases has become an urgent problem to be solved. Summary of the invention

[0004] In view of this, the present invention provides a test case collection method, device, electronic device and target vehicle to solve the problem of how to reuse test cases.

[0005] In a first aspect, the present invention provides a test case collection method, the method comprising:

[0006] Continuously monitor the functional test behavior of the target software; the target software is the vehicle-mounted voice interaction software; the functional test behavior is used to test the interaction between the target software and the target vehicle;

[0007] If the functional testing behavior of the target software is monitored, the dynamic logs of the functional testing behavior are captured in real time;

[0008] Identify dynamic logs and generate test cases corresponding to functional test behaviors;

[0009] Store test cases in a preset test case database to achieve reuse of test cases.

[0010] The test case collection method provided by the embodiment of the present application continuously monitors the functional test behavior for the target software. If the functional test behavior for the target software is monitored, each dynamic log in the process of executing the functional test behavior is captured in real time, so that the dynamic log can be identified and the test case corresponding to the functional test behavior is generated. The accuracy of the generated test case is guaranteed. The test case is stored in a preset test case database, so that the test case can be reused. The above method does not require manual collection or recording of test case data to generate test cases, but generates test cases corresponding to the functional test behavior by identifying the dynamic log. Therefore, the efficiency of generating test cases is improved, and the storage of test cases is realized, and then the test cases can be reused through the test cases stored in the preset test case database, thereby improving the testing efficiency for the vehicle-mounted voice interaction software and reducing labor costs.

[0011] In an optional implementation, the dynamic log is identified to generate a test case corresponding to the functional test behavior, including:

[0012] Identify the dynamic log, determine the expected behavior of the target vehicle and the target behavior of the target vehicle corresponding to the functional test behavior, and determine the test data corresponding to the functional test behavior;

[0013] Generate test cases corresponding to functional test behaviors based on expected behaviors, target behaviors, and test data.

[0014] The test case collection method provided in the embodiment of the present application identifies dynamic logs, determines the expected behavior of the target vehicle and the target behavior of the target vehicle corresponding to the functional test behavior, and determines the test data corresponding to the functional test behavior; based on the expected behavior, the target behavior and the test data, a test case corresponding to the functional test behavior is generated, thereby ensuring the accuracy of the generated test case and the comprehensiveness of the data.

[0015] In an optional implementation, generating a test case corresponding to the functional test behavior based on the expected behavior, the target behavior and the test data includes:

[0016] If the functional test behavior for the target software is monitored, the target vehicle is monitored to obtain the actual behavior of the target vehicle based on the functional test behavior;

[0017] Compare actual behavior to target behavior;

[0018] If the actual behavior is consistent with the target behavior, a test case corresponding to the functional test behavior is generated based on the expected behavior, the target behavior and the test data.

[0019] The test case collection method provided in the embodiment of the present application monitors the target vehicle if the functional test behavior for the target software is monitored; the actual behavior of the target vehicle based on the functional test behavior is obtained, thereby ensuring the accuracy of the actual behavior of the target vehicle based on the functional test behavior. The actual behavior is compared with the target behavior; if the actual behavior is consistent with the target behavior, a test case corresponding to the functional test behavior is generated based on the expected behavior and the target behavior and the test data, thereby ensuring the accuracy of the generated test case. This avoids inaccurate target behavior saved in the test case, resulting in inaccurate saved test case data, and thus inaccurate results of testing using the test case.

[0020] In an optional implementation, after generating the test case corresponding to the functional test behavior, the method further includes:

[0021] Identify target behaviors and expected behaviors;

[0022] If the target behavior is consistent with the expected behavior, the generated test case is determined to be a successful test case;

[0023] If the target behavior is inconsistent with the expected behavior, the generated test case is determined to be a failed test case.

[0024] The test case collection method provided in the embodiment of the present application identifies the target behavior and the expected behavior; if the target behavior is consistent with the expected behavior, the generated test case is determined to be a successful test case, ensuring the accuracy of the generated test case being a successful test case. If the target behavior is inconsistent with the expected behavior, the generated test case is determined to be a failed test case, ensuring the accuracy of the generated test case being a failed test case.

[0025] In an optional implementation, the target software includes a plurality of sub-software modules, and after determining that the generated test case is a failed test case, the method further includes:

[0026] Identify dynamic logs to determine the actual and expected results of each sub-software module in the target software;

[0027] Compare the actual results and expected results corresponding to each sub-software module to determine the execution results corresponding to each sub-software module;

[0028] Determine the failure cause of the failed test case based on the execution results of each sub-software module;

[0029] Categorize failed test cases based on the reasons for failure.

[0030] The test case collection method provided in the embodiment of the present application identifies the dynamic log, determines the actual results and expected results output by each sub-software module in the target software, and ensures the accuracy of the actual results and expected results output by each sub-software module. Then, the actual results and expected results corresponding to each sub-software module are compared to determine the execution results corresponding to each sub-software module, and ensure the accuracy of the execution results corresponding to each sub-software module. According to the execution results corresponding to each sub-software module, the failure cause corresponding to the failed test case is determined, and the accuracy of the failure cause corresponding to the determined failed test case is ensured. According to the failure cause, the failed test case is classified, and the accuracy of the classification of the failed test case is ensured.

[0031] In an optional implementation, the target software includes at least one of a speech recognition module, a natural language understanding module, and a text-to-speech module, and the actual results corresponding to each sub-software module are compared with the expected results to determine the execution results corresponding to each sub-software module, including:

[0032] Obtaining the actual semantic text output by the speech recognition module and the expected semantic text corresponding to the speech recognition module;

[0033] Calculating a first similarity between the actual semantic text and the expected semantic text;

[0034] Determining, according to the first similarity, an execution result corresponding to the speech recognition module;

[0035] and / or,

[0036] Obtain the actual domain, actual intent, and actual slot output by the natural language understanding module; and the expected domain, expected intent, and expected slot corresponding to the speech recognition module;

[0037] Determine an execution result corresponding to the natural language understanding module according to a second similarity between the actual domain and the expected domain, a third similarity between the actual intent and the expected intent, and a fourth similarity between the actual slot and the expected slot;

[0038] and / or,

[0039] Get the actual answers output by the text-to-speech module and the expected questions input to the text-to-speech module;

[0040] Based on the semantic relevance between the actual answer and the expected question, the execution result corresponding to the text-to-speech module is determined.

[0041] The test case collection method provided in the embodiment of the present application obtains the actual semantic text output by the speech recognition module and the expected semantic text corresponding to the speech recognition module; calculates the first similarity between the actual semantic text and the expected semantic text; determines the execution result corresponding to the speech recognition module according to the first similarity, thereby ensuring the accuracy of the execution result corresponding to the determined speech recognition module. And / or, obtains the actual domain, actual intention and actual slot output by the natural language understanding module; and the expected domain, expected intention and expected slot corresponding to the speech recognition module; determines the execution result corresponding to the natural language understanding module according to the second similarity between the actual domain and the expected domain, the third similarity between the actual intention and the expected intention, and the fourth similarity between the actual slot and the expected slot, thereby ensuring the accuracy of the execution result corresponding to the determined natural language understanding module. And / or, obtains the actual answer output by the text-to-speech module and the expected question input to the text-to-speech module; determines the execution result corresponding to the text-to-speech module according to the semantic relevance between the actual answer and the expected question, thereby ensuring the accuracy of the execution result corresponding to the determined text-to-speech module.

[0042] In an optional implementation, determining the execution result corresponding to the text-to-speech module according to the semantic relevance between the actual answer and the expected question includes:

[0043] extracting at least one first keyword from the actual answer;

[0044] extracting at least one second keyword from the expected question;

[0045] Calculating lexical similarity and semantic relevance between each first keyword and each second keyword;

[0046] Determine the semantic relevance between the actual answer and the expected question based on lexical similarity and semantic relevance;

[0047] Based on the semantic relevance between the actual answer and the expected question, the execution result corresponding to the text-to-speech module is determined.

[0048] The test case collection method provided in the embodiment of the present application extracts at least one first keyword from the actual answer; extracts at least one second keyword from the expected question; calculates the lexical similarity and semantic relevance between each first keyword and each second keyword, thereby ensuring the accuracy of the lexical similarity and semantic relevance between each first keyword and each second keyword calculated. According to the lexical similarity and semantic relevance, the semantic relevance between the actual answer and the expected question is determined; according to the semantic relevance between the actual answer and the expected question, the execution result corresponding to the text-to-speech module is determined, thereby ensuring the accuracy of the execution result corresponding to the determined text-to-speech module.

[0049] In a second aspect, the present invention provides a test case collection device, the device comprising:

[0050] A monitoring module is used to continuously monitor the functional test behavior of the target software; the target software is the vehicle-mounted voice interaction software; the functional test behavior is used to test the interaction behavior between the target software and the target vehicle;

[0051] The capture module is used to capture various dynamic logs in the process of executing the functional test behavior in real time if the functional test behavior for the target software is monitored;

[0052] The first identification module is used to identify the dynamic log and generate test cases corresponding to the functional test behavior;

[0053] The storage module is used to store the test cases in a preset test case database to achieve the reuse of the test cases.

[0054] The test case collection device provided in the embodiment of the present application continuously monitors the functional test behavior for the target software. If the functional test behavior for the target software is monitored, the various dynamic logs in the process of executing the functional test behavior are captured in real time, so that the dynamic logs can be identified and the test cases corresponding to the functional test behavior are generated. The accuracy of the generated test cases is guaranteed. The test cases are stored in a preset test case database, so that the test cases can be reused. The above device does not require manual collection or recording of test case data to generate test cases, but generates test cases corresponding to the functional test behavior by identifying dynamic logs. Therefore, the efficiency of generating test cases is improved, and the storage of test cases is realized, and then the test cases can be reused through the test cases stored in the preset test case database, thereby improving the testing efficiency for the vehicle-mounted voice interaction software and reducing labor costs.

[0055] In a third aspect, the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the test case collection method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0056] In a fourth aspect, the present invention provides a target vehicle, comprising a vehicle body and the electronic device of the third aspect.

[0057] In a fifth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the test case collection method of the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 is a flow chart of a test case collection method according to an embodiment of the present invention;

[0060] Figure 2 is a flow chart of another test case collection method according to an embodiment of the present invention;

[0061] Figure 3 is a flowchart of another test case collection method according to an embodiment of the present invention;

[0062] Figure 4 is a structural block diagram of a test case collection device according to an embodiment of the present invention;

[0063] Figure 5 is a structural block diagram of another test case collection device according to an embodiment of the present invention;

[0064] Figure 6 is a structural block diagram of another test case collection device according to an embodiment of the present invention;

[0065] Figure 7 It is a schematic diagram of the hardware structure of the electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0067] Faced with the current situation of diversified car models and rapid iteration of in-vehicle voice interaction assistant versions, traditional functional testing methods for voice interaction assistants have shown their limitations. The main manifestations are low testing efficiency and high labor costs. Each iteration means starting from scratch or modifying the existing test process in large quantities. During the testing process, it seems difficult to cope with frequent changes, which not only consumes precious resources but also restricts the response speed of the test team.

[0068] Therefore, how to achieve the reuse of test cases has become an urgent problem to be solved.

[0069] The embodiment of the present application provides a test case collection method, which continuously monitors the functional test behavior for the target software. If the functional test behavior for the target software is monitored, the various dynamic logs in the process of executing the functional test behavior are captured in real time, so that the dynamic logs can be identified and the test cases corresponding to the functional test behavior are generated. The accuracy of the generated test cases is guaranteed. The test cases are stored in a preset test case database, so that the test cases can be reused. The above method does not require manual collection or recording of test case data to generate test cases, but generates test cases corresponding to the functional test behavior by identifying dynamic logs. Therefore, the efficiency of generating test cases is improved, and the storage of test cases is realized, and then the test cases can be reused through the test cases stored in the preset test case database, thereby improving the testing efficiency for the vehicle-mounted voice interaction software and reducing labor costs.

[0070] It should be noted that the method for collecting test cases provided in the embodiments of the present application, the execution subject of which can be a device for collecting test cases, can be implemented as part or all of an electronic device through software, hardware, or a combination of software and hardware, wherein the electronic device can be a control device in the target vehicle, or an electronic device independent of the target vehicle. When the electronic device is an electronic device independent of the target vehicle, the electronic device can be a server or a terminal, wherein the server in the embodiments of the present application can be a single server, or a server cluster composed of multiple servers, and the terminal in the embodiments of the present application can be a smart phone, a personal computer, a tablet computer, a wearable device, and other intelligent hardware devices such as an intelligent robot. In the following method embodiments, the execution subject is an electronic device as an example for explanation.

[0071] According to an embodiment of the present invention, a test case collection method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0072] In this embodiment, a test case collection method is provided, which can be used for the above-mentioned electronic device. Figure 1 is a flow chart of a test case collection method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0073] Step S101: continuously monitor the functional testing behavior of the target software.

[0074] Among them, the target software is the in-vehicle voice interaction software; the functional test behavior is used to test the interaction behavior between the target software and the target vehicle.

[0075] Specifically, the electronic device can continuously monitor the functional testing behavior for the target software by monitoring the test voice instructions input by the user.

[0076] If the electronic device monitors the test voice command input by the user, the electronic device determines that a functional test behavior for the target software is monitored.

[0077] Step S102: if a functional test behavior for the target software is monitored, then each dynamic log in the process of executing the functional test behavior is captured in real time.

[0078] Specifically, the electronic device can obtain various dynamic logs generated in the target software during the execution of the functional test behavior from the log recording module in the target software. These dynamic logs include the reception time, content, recognition results, semantic understanding process, called functional modules, data interacting with the vehicle system, and execution results of the voice command. For example, when a user issues a voice command of "navigate to the city center", the log should record the precise timestamp of the command reception, the process of voice recognition as "navigate to the city center", the steps of semantic understanding to determine it as a navigation function and resolve the destination as "city center", and the subsequent interaction with the navigation system to obtain information on the execution process such as route planning.

[0079] The electronic device can obtain various dynamic logs from the target vehicle during the process of executing the test actions corresponding to the functional test behavior. These logs record the changes in the state of the target vehicle itself and the information of interaction with the target software, such as the changes in vehicle sensor data (vehicle speed, position, direction, etc.) during the functional test of the target software, and the response of the vehicle system to the target software (such as the route planning calculation after the navigation system receives the destination information, the operation of the multimedia system to play music according to voice commands, etc.). Through the coordinated recording of the logs of both the target software and the target vehicle system, various dynamic information in the process of functional test behavior can be fully reflected.

[0080] Step S103: Identify the dynamic log and generate a test case corresponding to the functional test behavior.

[0081] Specifically, electronic devices can use real-time data analysis algorithms to analyze captured dynamic logs to promptly identify abnormal situations, performance bottlenecks, and potential problems in functional test behaviors.

[0082] For example, real-time stream processing technology (such as Apache Flink, etc.) is used to perform real-time calculation and analysis on dynamic log data streams, and abnormal events (such as a sudden drop in voice recognition accuracy, excessive vehicle system response time, etc.) are detected by setting thresholds, pattern matching, etc. At the same time, statistical analysis methods (such as mean and variance calculations, etc.) are combined to monitor and evaluate the performance indicators of the target software and target vehicle system in real time, such as calculating the average time for voice command processing, the success rate of the vehicle system's response to voice commands, etc., and based on the calculation and analysis results, test cases corresponding to the functional test behavior are generated.

[0083] This step will be described in detail below.

[0084] Step S104, storing the test case in a preset test case database to achieve the reuse of the test case.

[0085] Specifically, the electronic device stores the test cases in a preset test case database according to a preset storage path to achieve reuse of the test cases.

[0086] The test case collection method provided in this embodiment continuously monitors the functional test behavior for the target software. If the functional test behavior for the target software is monitored, the various dynamic logs in the process of executing the functional test behavior are captured in real time, so that the dynamic logs can be identified and the test cases corresponding to the functional test behavior are generated. The accuracy of the generated test cases is guaranteed. The test cases are stored in a preset test case database, so that the test cases can be reused. The above method does not require manual collection or recording of test case data to generate test cases, but generates test cases corresponding to the functional test behavior by identifying dynamic logs. Therefore, the efficiency of generating test cases is improved, and the storage of test cases is realized, and then the test cases can be reused through the test cases stored in the preset test case database, thereby improving the testing efficiency of the vehicle-mounted voice interaction software and reducing labor costs.

[0087] In this embodiment, a test case collection method is provided, which can be used for the above-mentioned electronic device. Figure 2 is a flow chart of a test case collection method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0088] Step S201: continuously monitor the functional testing behavior of the target software.

[0089] Among them, the target software is the in-vehicle voice interaction software; the functional test behavior is used to test the interaction behavior between the target software and the target vehicle.

[0090] For details about this step, please refer to the above description of step S101, which will not be elaborated here.

[0091] Step S202: if a functional test behavior for the target software is monitored, then each dynamic log in the process of executing the functional test behavior is captured in real time.

[0092] For details about this step, please refer to the above description of step S102, which will not be elaborated here.

[0093] Step S203: Identify the dynamic log and generate a test case corresponding to the functional test behavior.

[0094] Specifically, the above step S203 may include the following steps:

[0095] Step S2031, identifying the dynamic log, determining the expected behavior of the target vehicle and the target behavior of the target vehicle corresponding to the functional test behavior, and determining the test data corresponding to the functional test behavior.

[0096] Specifically, the electronic device can perform structured processing on the captured dynamic log, convert it into a format that is easy to analyze, identify the converted dynamic log, and determine the log events included in the dynamic log. For example, in a table format, each row represents a log event, and each column contains different attributes of the event (such as timestamp, event type, related parameters, etc.). For example, for a log recording the speed change of the target vehicle, it can be expressed as [timestamp, "vehicle speed change", speed value]. In this way, the unstructured log data is organized into a data set with a clear structure, which is convenient for subsequent query and analysis.

[0097] For example, the parameter examples may be shown in the following table:

[0098]

[0099] Then, the electronic device classifies and labels each log event in the dynamic log according to its nature and source. For example, the logs related to voice commands are marked as "voice interaction", including specific labels such as voice recognition start, recognition results, and semantic understanding results; the logs related to vehicle system responses are marked as "vehicle control", including labels such as vehicle speed adjustment, direction rotation, and device status changes; the logs related to the test environment are marked as "environmental information", such as network connection status, sensor working status, etc. Such classification and labeling help to quickly filter and locate log data related to functional test behavior.

[0100] Then, based on the understanding of the target software functions and the target vehicle control logic, the electronic device establishes a library of expected behavior patterns under various functional test scenarios, and then searches for the expected behavior corresponding to the test voice command from the expected behavior pattern according to the test voice command input by the user. For example, for the "navigate to destination" functional test scenario, the expected behavior pattern includes a series of behaviors such as the vehicle voice interaction software correctly identifying and parsing the destination after the voice command is issued, and then sending a request to the navigation system, the navigation system calculating the route and displaying the navigation information on the vehicle display, and the vehicle adjusting the driving direction according to the navigation command. Each expected behavior pattern is defined in the form of a log event sequence and related parameter ranges.

[0101] During the execution of the functional test behavior, the electronic device identifies the log data generated in real time in the dynamic log, and determines the target behavior of the target vehicle corresponding to the functional test behavior.

[0102] In addition, the electronic device also needs to identify the log data generated in real time in the dynamic log, and extract the test data corresponding to the functional test behavior from it. These test data include but are not limited to the content of the voice command, the expected ASR result, the expected semantic understanding result, the expected TTS reply, the execution result, the ASR recognition result, the semantic understanding result, the TTS reply result, the command execution result, the vehicle system response time, the parameters of the execution operation (such as the navigation destination coordinates, the value of adjusting the vehicle speed, etc.), the vehicle control parameters (such as the vehicle speed, the gear information, etc.), the chassis system parameters (such as the steering angle, the suspension state, etc.), the vehicle environment parameters (such as: vehicle positioning, camera data, etc.), the multimedia system parameters (such as: audio playback status, video playback status, etc.), the in-vehicle environment parameters (such as: air conditioning system parameters, seat adjustment parameters, etc.) and the vehicle communication system parameters (such as: Bluetooth connection status, wifi connection status) and the test environment data (such as temperature, humidity, network signal strength, etc.). For example, for the voice command "turn on the air conditioner to 25 degrees", the key test data extracted include the original voice command, the temperature setting value (25 degrees) received by the air conditioning control module, and the time to execute the operation.

[0103] Step S2032: Generate a test case corresponding to the functional test behavior based on the expected behavior, the target behavior and the test data.

[0104] In an optional implementation, the electronic device cleans the test data corresponding to the test case, including removing duplicate data, correcting erroneous data, filling in missing data, etc., thereby better ensuring the neatness and completeness of the test case data.

[0105] The cleaned test data is associated and integrated to form a complete test case corresponding to the functional test behavior. Test data from different sources and types are associated through unique identifiers such as timestamps and event IDs to ensure data integrity and consistency. For example, the time when the voice command is issued is associated with the time when the vehicle air conditioning system actually starts to adjust the temperature, and the delay time between the two is calculated as an important test data for evaluating the response speed of the voice interaction control air conditioning function. At the same time, the test data is associated with the corresponding expected behavior and target behavior so that all aspects of the functional test behavior can be fully understood when analyzing the test results.

[0106] Exemplarily, the generated test case data table may be shown in Table 2:

[0107]

[0108] In another optional implementation, the above step S2032 may include the following steps:

[0109] Step a1: If a functional test behavior for the target software is monitored, the target vehicle is monitored to obtain the actual behavior of the target vehicle based on the functional test behavior.

[0110] Specifically, if the functional test behavior for the target software is monitored, the electronic device can collect the behavior data corresponding to the target vehicle based on the communication connection with each sensor on the target vehicle. Optionally, the electronic device can also obtain the behavior data corresponding to the target vehicle collected by each sensor sent by the ECU based on the communication connection with the ECU in the target vehicle. Among them, these sensors include but are not limited to vehicle speed sensors, steering wheel angle sensors, gear sensors, brake sensors, throttle sensors, vehicle position sensors (such as GPS), in-vehicle environment sensors (temperature, humidity, etc.) and sensors such as cameras and radars for obtaining information about the vehicle's surrounding environment.

[0111] The electronic device reads the data from each sensor in real time and converts it into a unified data format for subsequent processing. For example, the vehicle speed sensor converts the wheel rotation signal into a digital vehicle speed value, and the data acquisition module collects this data at a fixed frequency (such as 100Hz) and adds a timestamp.

[0112] Then, the data from each sensor is integrated to determine the actual behavior of the target vehicle under the functional test behavior. For example, by fusing the data from the steering wheel angle sensor, vehicle speed sensor, and vehicle position sensor, the trajectory and posture changes of the target vehicle during driving can be accurately depicted; combined with camera and radar data, the actual behavior of the target vehicle in responding to traffic conditions and executing voice commands (such as changing lanes, avoiding obstacles, etc.) can also be analyzed. Through this multi-source data fusion method, more comprehensive and accurate information on the actual behavior of the target vehicle can be obtained than single sensor data.

[0113] Step a2, compare the actual behavior with the target behavior.

[0114] Specifically, the electronic device compares the actual behavior with the target behavior in the dynamic log.

[0115] Step a3: If the actual behavior is consistent with the target behavior, a test case corresponding to the functional test behavior is generated based on the expected behavior, the target behavior and the test data.

[0116] Specifically, if the actual behavior is consistent with the target behavior, the electronic device generates a test case corresponding to the functional test behavior based on the expected behavior, the target behavior and the test data.

[0117] If the actual behavior is consistent with the target behavior, the electronic device determines that the functional test behavior has failed. Therefore, it is prohibited to generate a test case corresponding to the functional test behavior based on the expected behavior, the target behavior and the test data.

[0118] Step S204, storing the test case in a preset test case database to achieve the reuse of the test case.

[0119] For details about this step, please refer to the above description of step S104, which will not be elaborated here.

[0120] The test case collection method provided in the embodiment of the present application identifies the dynamic log, determines the expected behavior of the target vehicle and the target behavior of the target vehicle corresponding to the functional test behavior, and determines the test data corresponding to the functional test behavior; if the functional test behavior for the target software is monitored, the target vehicle is monitored; the actual behavior of the target vehicle based on the functional test behavior is obtained, and the accuracy of the actual behavior of the target vehicle based on the functional test behavior is ensured. The actual behavior is compared with the target behavior; if the actual behavior is consistent with the target behavior, a test case corresponding to the functional test behavior is generated based on the expected behavior and the target behavior and the test data, and the accuracy of the generated test case is ensured. It avoids the inaccuracy of the target behavior saved in the test case, resulting in inaccurate test case data saved, and thus inaccurate results of testing using the test case.

[0121] In an optional embodiment of the present application, Figure 3 As shown, after the "generating test cases corresponding to the functional test behaviors" in the above step S203 and / or step S2032 and / or step a4, the following steps may be included:

[0122] Step S301, identifying the target behavior and the expected behavior.

[0123] Specifically, the electronic device identifies and compares the actual behavior of the target vehicle saved in the test case with the expected behavior.

[0124] Step S302: If the target behavior is consistent with the expected behavior, the generated test case is determined to be a successful test case.

[0125] Specifically, if the target behavior is consistent with the expected behavior, the generated test case is determined to be a successful test case.

[0126] Step S303: If the target behavior is inconsistent with the expected behavior, the generated test case is determined to be a failed test case.

[0127] Specifically, if the target behavior is inconsistent with the expected behavior, the generated test case is determined to be a failed test case.

[0128] Step S304, identifying the dynamic log, and determining the actual results and expected results output by each sub-software module in the target software.

[0129] Specifically, after determining that the generated test case is a failed test case, the electronic device can identify the dynamic log generated in the target software and determine the actual results and expected results output by each sub-software module in the target software.

[0130] Step S305 , comparing the actual result corresponding to each sub-software module with the expected result, and determining the execution result corresponding to each sub-software module.

[0131] Specifically, the target software includes at least one of a speech recognition module, a natural language understanding module, and a text-to-speech module. The above step S305 may include the following steps:

[0132] Step S3051, obtaining the actual semantic text output by the speech recognition module and the expected semantic text corresponding to the speech recognition module.

[0133] Specifically, the electronic device can identify the dynamic log generated in the target software, and determine the actual semantic text output by the speech recognition module and the expected semantic text corresponding to the speech recognition module.

[0134] Step S3052: Calculate the first similarity between the actual semantic text and the expected semantic text.

[0135] Specifically, the electronic device can segment the actual semantic text and the expected semantic text according to the preset segmentation rules. For example, for English text, the electronic device can use the WordPiece segmentation algorithm to split the words into sub-word units. For Chinese text, the electronic device can use a special Chinese segmentation tool (such as Jieba, etc.) for segmentation.

[0136] Then, the electronic device adds special tags, such as [CLS] (classification tag, used to represent the semantics of the entire sentence) and [SEP] (separation tag, used to separate different sentences or text fragments), at the beginning and end of the actual semantic text and the expected semantic text.

[0137] The electronic device converts the actual semantic text after word segmentation and the expected semantic text into a tensor format. It is usually a sequence of integer indices, where each integer corresponds to a word or subword in the vocabulary. If the text lengths are inconsistent, padding or truncation can be performed to make its length meet the input requirements of the model.

[0138] Then, the electronic device can encode the actual semantic text and the expected semantic text converted into tensor format into the pre-trained semantic understanding model. The pre-trained semantic understanding model will perform a series of processing on the actual semantic text and the expected semantic text converted into tensor format, including:

[0139] The integer index sequences corresponding to the actual semantic text and the expected semantic text of the input are converted into low-dimensional vector representations, and each word or subword is mapped to a vector space of fixed dimension. These vectors capture the semantic and grammatical information of the words.

[0140] In order to represent the position information of words in sentences, the pre-trained semantic understanding model adds position embedding. The position embedding is added to the word embedding, so that the pre-trained semantic understanding model can distinguish words in different positions and thus understand the order and structure of sentences.

[0141] Then, the embedded actual semantic text and expected semantic text are encoded through multiple Transformer layers to obtain the encoded actual encoding vector and expected encoding vector. Among them, the Transformer layer contains a multi-head attention mechanism, which can simultaneously focus on different parts of the input actual semantic text and the expected semantic text, capturing the semantic relationship and contextual information between words. In each Transformer layer, the representation of the text is continuously updated and refined, and more advanced semantic features are gradually learned.

[0142] After encoding processing by the pre-trained semantic understanding model, the actual semantic text and the expected semantic text are converted into actual encoding vectors and expected encoding vectors of fixed dimensions respectively.

[0143] The electronic device may calculate the semantic similarity between the actual coding vector and the expected coding vector based on the following formula.

[0144]

[0145] Among them, Vec1=Bert_Embedding is the actual coding vector, and Vec2=Bert_Embedding is the expected coding vector.

[0146] The electronic device may also calculate the edit distance similarity between the actual encoding vector and the expected encoding vector.

[0147] Then, the electronic device adds the semantic similarity between the actual coding vector and the expected coding vector and the edit distance similarity between the actual coding vector and the expected coding vector, and calculates the first similarity between the actual semantic text and the expected semantic text.

[0148] Step S3053: Determine the execution result corresponding to the speech recognition module according to the first similarity.

[0149] Specifically, the electronic device compares the first similarity with the first preset similarity threshold. If the first similarity is greater than or equal to the first preset similarity threshold, it determines that the execution result corresponding to the voice recognition module is a successful execution; if the first similarity is less than the first preset similarity threshold, it determines that the execution result corresponding to the voice recognition module is a failed execution.

[0150] The first preset similarity threshold may be 95% or 90%. The embodiment of the present application does not specifically limit the first preset similarity threshold.

[0151] and / or,

[0152] Step S3054, obtaining the actual domain, actual intention and actual slot output by the natural language understanding module; and the expected domain, expected intention and expected slot corresponding to the speech recognition module.

[0153] Specifically, the electronic device can identify the dynamic logs generated in the target software, determine the actual domain, actual intention and actual slot output by the natural language understanding module; and the expected domain, expected intention and expected slot corresponding to the speech recognition module.

[0154] Step S3055, determining the execution result corresponding to the natural language understanding module according to the second similarity between the actual domain and the expected domain, the third similarity between the actual intention and the expected intention, and the fourth similarity between the actual slot and the expected slot.

[0155] Optionally, the electronic device may use a preset similarity calculation algorithm to calculate the second similarity between the actual domain and the expected domain, the third similarity between the actual intention and the expected intention, and the fourth similarity between the actual slot and the expected slot. Then, the weight information of the second similarity, the third similarity, and the fourth similarity is obtained respectively. The electronic device multiplies the second similarity, the third similarity, and the fourth similarity by the corresponding weight information, respectively, calculates the target similarity, and determines the execution result corresponding to the natural language understanding module according to the target similarity.

[0156] The electronic device compares the target similarity with the target preset similarity threshold. If the target similarity is greater than or equal to the target preset similarity threshold, the execution result corresponding to the natural language understanding module is determined to be a successful execution; if the target similarity is less than the target preset similarity threshold, the execution result corresponding to the natural language understanding module is determined to be a failed execution.

[0157] The target preset similarity threshold may be 95% or 90%. The embodiment of the present application does not specifically limit the target preset similarity threshold.

[0158] Optionally, the electronic device may also first calculate the second similarity between the actual domain and the expected domain. If the second similarity is less than the second preset similarity threshold, the execution result corresponding to the natural language understanding module is determined to be execution failure; if the second similarity is greater than or equal to the second preset similarity threshold, the third similarity between the actual intention and the expected intention is calculated again; if the third similarity is less than the third preset similarity threshold, the execution result corresponding to the natural language understanding module is determined to be execution failure; if the third similarity is greater than or equal to the third preset similarity threshold, the fourth similarity between the actual slot and the expected slot is calculated again; if the fourth similarity is less than the fourth preset similarity threshold, the execution result corresponding to the natural language understanding module is determined to be execution failure; if the fourth similarity is greater than or equal to the fourth preset similarity threshold, the execution result corresponding to the natural language understanding module is determined to be execution success.

[0159] and / or,

[0160] Step S3056, obtaining the actual answer output by the text-to-speech module and the expected question input to the text-to-speech module.

[0161] Specifically, the electronic device can identify the dynamic logs generated in the target software, determine the actual answer output by the text-to-speech module, and the expected question input to the text-to-speech module.

[0162] Step S3057, determine the execution result corresponding to the text-to-speech module according to the semantic relevance between the actual answer and the expected question.

[0163] In an alternative embodiment of the present application.

[0164] Specifically, the above step S3057 may include the following steps:

[0165] Step b1, extract at least one first keyword from the actual answer.

[0166] Specifically, the electronic device can first clean the actual answer output by the text-to-speech module, removing punctuation marks, stop words (such as common words with little contribution to semantic judgment like "of", "is", "in", etc.), and special characters, etc. For example, processing "The weather is really nice today!" into "The weather is really nice today". Then, the electronic device can use stemming or lemmatization techniques to convert words into their basic forms. For example, converting "running" to "run" and "children" to "child" for better keyword matching and semantic analysis.

[0167] Next, the electronic device can use a statistical method: calculate the term frequency (TF) of each word in the actual answer, and at the same time combine the inverse document frequency (IDF) to measure the importance of the word. Words with higher TF-IDF values are more likely to be keywords. For example, in an actual answer about the introduction of tourist attractions, if the TF-IDF value of the word "The Forbidden City" is high, then it is very likely to be a key information point and is extracted as the first keyword.

[0168] The electronic device can use a pre-trained word vector model (such as Word2Vec, GloVe, etc.) to convert the words in the actual answer into vector representations. Then, through a clustering algorithm (such as K-Means clustering), the vectors are clustered, and the words near each cluster center are selected as keywords, so as to obtain at least one first keyword corresponding to the actual answer. For example, for an actual answer about the introduction of a technology product, the neural network model may assign higher weights to words related to the core features of the product such as "chip performance" and "screen resolution", and thus extract them as the first keywords.

[0169] Step b2, extract at least one second keyword from the expected question.

[0170] Specifically, the electronic device can refer to the method of "extracting at least one first keyword from the actual answer" in the above step b1 to extract at least one second keyword from the expected question, which will not be elaborated here.

[0171] Step b3, calculating the lexical similarity and semantic relevance between each first keyword and each second keyword.

[0172] Specifically, the electronic device can calculate the edit distance between the first keyword and the second keyword, that is, the minimum number of editing operations (insertion, deletion, replacement of characters) required to convert from one word to another. The smaller the edit distance, the higher the lexical similarity. For example, the edit distance between "apple" and "Apple Inc." is small, indicating that they are closer in lexical form. However, the edit distance method may ignore the differences at the semantic level, so it needs to be combined with other methods.

[0173] Then, the electronic device converts the first keyword and the second keyword into vector representations using a pre-trained word vector model, and then calculates the cosine similarity between the vectors.

[0174] The electronic device may perform weighted addition of the edit distances and cosine similarities corresponding to the first keywords and the second keywords to obtain the vocabulary similarities between each first keyword and each second keyword.

[0175] Then, the electronic device searches for the first keyword and the second keyword in a semantic knowledge graph (such as Baidu Encyclopedia Knowledge Graph, Freebase, etc.) and analyzes the semantic relationship path and connection strength between them. For example, if the first keyword is "car" and the second keyword is "engine", they may be connected through a "component" relationship in the knowledge graph, indicating that the semantic relevance is strong.

[0176] Then, the electronic device uses a pre-trained semantic relevance model (such as a semantic matching model based on BERT) to input the text pair consisting of the first keyword and the second keyword into the pre-trained semantic relevance model, and the pre-trained semantic relevance model outputs the semantic relevance between each first keyword and each second keyword. For example, for a keyword pair such as "doctor" and "hospital", the model can accurately determine that they have a strong semantic relevance because doctors usually work in hospitals.

[0177] Step b4, determining the semantic relevance between the actual answer and the expected question based on lexical similarity and semantic relevance.

[0178] Specifically, the electronic device can set weights for lexical similarity and semantic relevance respectively. Then, the lexical similarity and semantic relevance between each first keyword and each second keyword are multiplied by the weight of the response and then added to obtain the semantic relevance between the actual answer and the expected question.

[0179] For example, in a question-answering system with high requirements for semantic accuracy, the weight of semantic relevance can be set to 0.7, and the weight of lexical similarity can be set to 0.3. Then, the lexical similarity score and semantic relevance score of each first keyword and second keyword pair are weighted and summed to obtain a comprehensive score for each keyword pair. The comprehensive scores of all keyword pairs are summarized, such as taking the average or summing (selecting the appropriate summing method according to the specific situation), to obtain the overall semantic relevance score between the actual answer and the expected question.

[0180] Step b5, determining the execution result corresponding to the text-to-speech module according to the semantic relevance between the actual answer and the expected question.

[0181] Specifically, the electronic device can compare the semantic correlation between the actual answer and the expected question with a preset semantic correlation threshold. If the semantic correlation between the actual answer and the expected question is greater than or equal to the preset semantic correlation threshold, the execution result corresponding to the text-to-speech module is determined to be a successful execution; if the semantic correlation between the actual answer and the expected question is less than the preset semantic correlation threshold, the execution result corresponding to the text-to-speech module is determined to be a failed execution.

[0182] Step S306, determining the failure cause corresponding to the failed test case according to the execution result corresponding to each sub-software module.

[0183] Specifically, the electronic device determines the execution results corresponding to each sub-software module, and determines the target sub-software module whose execution result is execution failure from each sub-software module. Then, the failure reason corresponding to the failed test case is the execution failure of the target sub-software module.

[0184] Step S307, classifying failed test cases according to failure reasons.

[0185] Specifically, the electronic device classifies the failed test cases according to the failure reasons corresponding to the failed test cases.

[0186] The test case collection method provided in the embodiment of the present application identifies the target behavior and the expected behavior; if the target behavior is consistent with the expected behavior, the generated test case is determined to be a successful test case, ensuring the accuracy of the generated test case being a successful test case. If the target behavior is inconsistent with the expected behavior, the generated test case is determined to be a failed test case, ensuring the accuracy of the generated test case being a failed test case.

[0187] Identify the dynamic log, determine the actual results and expected results output by each sub-software module in the target software, and ensure the accuracy of the actual results and expected results output by each sub-software module. Then, obtain the actual semantic text output by the speech recognition module and the expected semantic text corresponding to the speech recognition module; calculate the first similarity between the actual semantic text and the expected semantic text; determine the execution result corresponding to the speech recognition module according to the first similarity, and ensure the accuracy of the execution result corresponding to the determined speech recognition module. And / or, obtain the actual domain, actual intention and actual slot output by the natural language understanding module; and the expected domain, expected intention and expected slot corresponding to the speech recognition module; determine the execution result corresponding to the natural language understanding module according to the second similarity between the actual domain and the expected domain, the third similarity between the actual intention and the expected intention, and the fourth similarity between the actual slot and the expected slot, and ensure the accuracy of the execution result corresponding to the determined natural language understanding module. And / or, obtain the actual answer output by the text-to-speech module and the expected question input to the text-to-speech module; extract at least one first keyword from the actual answer; extract at least one second keyword from the expected question; calculate the lexical similarity and semantic relevance between each first keyword and each second keyword, and ensure the accuracy of the lexical similarity and semantic relevance between each first keyword and each second keyword calculated. According to the lexical similarity and semantic relevance, determine the semantic relevance between the actual answer and the expected question; according to the semantic relevance between the actual answer and the expected question, determine the execution result corresponding to the text-to-speech module, and ensure the accuracy of the execution result corresponding to the determined text-to-speech module. According to the execution results corresponding to each sub-software module, determine the failure cause corresponding to the failed test case, and ensure the accuracy of the failure cause corresponding to the determined failed test case. According to the failure cause, classify the failed test case, and ensure the accuracy of the classification of the failed test case.

[0188] In this embodiment, a test case collection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0189] This embodiment provides a test case collection device, such as Figure 4 As shown, including:

[0190] The monitoring module 401 is used to continuously monitor the functional test behavior of the target software; the target software is the vehicle-mounted voice interaction software; the functional test behavior is used to test the interaction behavior between the target software and the target vehicle;

[0191] The capture module 402 is used to capture various dynamic logs in the process of executing the functional test behavior in real time if the functional test behavior for the target software is monitored;

[0192] The first identification module 403 is used to identify the dynamic log and generate a test case corresponding to the functional test behavior;

[0193] The storage module 404 is used to store the test cases in a preset test case database to achieve the reuse of the test cases.

[0194] In some optional embodiments, the above-mentioned first identification module 403 is specifically used to identify dynamic logs, determine the expected behavior of the target vehicle and the target behavior of the target vehicle corresponding to the functional test behavior, and determine the test data corresponding to the functional test behavior; based on the expected behavior, the target behavior and the test data, generate a test case corresponding to the functional test behavior.

[0195] In some optional embodiments, the above-mentioned first identification module 403 is specifically used to monitor the target vehicle if functional testing behavior for the target software is monitored; obtain the actual behavior of the target vehicle based on the functional testing behavior; compare the actual behavior with the target behavior; if the actual behavior is consistent with the target behavior, generate a test case corresponding to the functional testing behavior based on the expected behavior, the target behavior and the test data.

[0196] In some optional implementations, the test case collection device, such as Figure 5 As shown, it also includes:

[0197] The second identification module 405 is used to identify the target behavior and the expected behavior;

[0198] A first determination module 406, configured to determine that the generated test case is a successful test case if the target behavior is consistent with the expected behavior;

[0199] The second determination module 407 is used to determine that the generated test case is a failed test case if the target behavior is inconsistent with the expected behavior.

[0200] In some optional implementations, the target software includes multiple sub-software modules, and the test case collection device, such as Figure 6 As shown, it also includes:

[0201] The third identification module 408 is used to identify the dynamic log and determine the actual results and expected results output by each sub-software module in the target software;

[0202] The comparison module 409 is used to compare the actual result and the expected result corresponding to each sub-software module to determine the execution result corresponding to each sub-software module;

[0203] The third determination module 410 is used to determine the failure cause corresponding to the failed test case according to the execution results corresponding to each sub-software module;

[0204] The classification module 411 is used to classify failed test cases according to the failure reasons.

[0205] In some optional embodiments, the target software includes at least one of a speech recognition module, a natural language understanding module, and a text-to-speech module. The above-mentioned comparison module 409 is specifically used to obtain the actual semantic text output by the speech recognition module and the expected semantic text corresponding to the speech recognition module; calculate the first similarity between the actual semantic text and the expected semantic text; determine the execution result corresponding to the speech recognition module based on the first similarity; and / or, obtain the actual domain, actual intention and actual slot output by the natural language understanding module; and the expected domain, expected intention and expected slot corresponding to the speech recognition module; determine the execution result corresponding to the natural language understanding module based on the second similarity between the actual domain and the expected domain, the third similarity between the actual intention and the expected intention, and the fourth similarity between the actual slot and the expected slot; and / or, obtain the actual answer output by the text-to-speech module and the expected question input to the text-to-speech module; determine the execution result corresponding to the text-to-speech module based on the semantic relevance between the actual answer and the expected question.

[0206] In some optional embodiments, the above-mentioned comparison module 409 is specifically used to extract at least one first keyword from the actual answer; extract at least one second keyword from the expected question; calculate the lexical similarity and semantic relevance between each first keyword and each second keyword; determine the semantic relevance between the actual answer and the expected question based on the lexical similarity and semantic relevance; determine the corresponding execution result of the text-to-speech module based on the semantic relevance between the actual answer and the expected question.

[0207] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0208] The test case collection device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0209] The embodiment of the present invention also provides an electronic device having the above Figure 4-Figure 6 The test case collection device shown.

[0210] See also Figure 7 , Figure 7 is a schematic diagram of the structure of an electronic device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the electronic device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0211] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0212] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0213] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0214] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0215] The electronic device further comprises a communication interface 30 for the electronic device to communicate with other devices or a communication network.

[0216] An embodiment of the present invention further provides a target vehicle, comprising a vehicle body and the electronic device of the above embodiment.

[0217] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0218] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0219] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A test case collection method, characterized in that: The method comprises: Continuously monitoring the functional test behavior of the target software; the target software is the vehicle-mounted voice interaction software; the functional test behavior is used to test the interaction behavior between the target software and the target vehicle; If a functional test behavior for the target software is monitored, each dynamic log in the process of executing the functional test behavior is captured in real time; Identify the dynamic log and generate a test case corresponding to the functional test behavior; The test cases are stored in a preset test case database to achieve the reuse of the test cases.

2. The method according to claim 1, characterized in that The identifying the dynamic log and generating a test case corresponding to the functional test behavior includes: Identify the dynamic log, determine the expected behavior of the target vehicle and the target behavior of the target vehicle corresponding to the functional test behavior, and determine the test data corresponding to the functional test behavior; Based on the expected behavior, the target behavior and the test data, a test case corresponding to the functional test behavior is generated.

3. The method according to claim 2, characterized in that The generating a test case corresponding to the functional test behavior based on the expected behavior, the target behavior and the test data includes: If a functional test behavior for the target software is monitored, the target vehicle is monitored to obtain an actual behavior of the target vehicle based on the functional test behavior; comparing the actual behavior to the target behavior; If the actual behavior is consistent with the target behavior, a test case corresponding to the functional test behavior is generated based on the expected behavior, the target behavior and the test data.

4. The method according to any one of claims 2 to 3, characterized in that: After generating the test case corresponding to the functional test behavior, the method further includes: identifying the target behavior and the expected behavior; If the target behavior is consistent with the expected behavior, determining that the generated test case is a successful test case; If the target behavior is inconsistent with the expected behavior, the generated test case is determined to be a failed test case.

5. The method according to claim 4, characterized in that The target software includes a plurality of sub-software modules. After determining that the generated test case is a failed test case, the method further includes: Identify the dynamic log to determine the actual results and expected results output by each sub-software module in the target software; Compare the actual result and the expected result corresponding to each of the sub-software modules to determine the execution result corresponding to each of the sub-software modules; Determine the failure cause of the failed test case according to the execution results of each of the sub-software modules; The failed test cases are classified according to the failure reasons.

6. The method according to claim 5, characterized in that The target software includes at least one of a speech recognition module, a natural language understanding module, and a text-to-speech module, and the actual results corresponding to each of the sub-software modules are compared with the expected results to determine the execution results corresponding to each of the sub-software modules, including: Acquire the actual semantic text output by the speech recognition module and the expected semantic text corresponding to the speech recognition module; Calculating a first similarity between the actual semantic text and the expected semantic text; Determining, according to the first similarity, an execution result corresponding to the speech recognition module; and / or, Obtaining the actual domain, actual intention, and actual slot output by the natural language understanding module; and the expected domain, expected intention, and expected slot corresponding to the speech recognition module; Determining an execution result corresponding to the natural language understanding module according to a second similarity between the actual domain and the expected domain, a third similarity between the actual intent and the expected intent, and a fourth similarity between the actual slot and the expected slot; and / or, obtaining actual answers output by the text-to-speech module and expected questions input to the text-to-speech module; According to the semantic relevance between the actual answer and the expected question, an execution result corresponding to the text-to-speech module is determined.

7. The method according to claim 6, characterized in that The determining, according to the semantic relevance between the actual answer and the expected question, the execution result corresponding to the text-to-speech module comprises: extracting at least one first keyword from the actual answer; extracting at least one second keyword from the expected question; Calculating lexical similarity and semantic relevance between each of the first keywords and each of the second keywords; Determining the semantic relevance between the actual answer and the expected question based on the lexical similarity and the semantic relevance; According to the semantic relevance between the actual answer and the expected question, an execution result corresponding to the text-to-speech module is determined.

8. A test case collection device, characterized in that: The device comprises: A monitoring module, used for continuously monitoring a functional test behavior for a target software; the target software is an in-vehicle voice interaction software; the functional test behavior is used for testing an interaction behavior between the target software and a target vehicle; A capture module is used to capture in real time various dynamic logs during the execution of the functional test behavior if a functional test behavior for the target software is monitored; A first identification module, used to identify the dynamic log and generate a test case corresponding to the functional test behavior; The storage module is used to store the test cases in a preset test case database to achieve the reuse of the test cases.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the test case collection method according to any one of claims 1 to 7 by executing the computer instructions.

10. A target vehicle, characterized in that: The invention comprises a vehicle body and the electronic device according to claim 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the test case collection method according to any one of claims 1 to 7.

Images