Vehicle voice interaction system test method and device, electronic equipment and storage medium

Abstract

The invention provides a vehicle voice interaction system test method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a target reference dictionary; performing mapping processing by using a preset intention label and the target reference dictionary to obtain a target mapping result; performing test processing on the vehicle voice interaction system based on the target mapping result to obtain a test execution result; and analyzing and processing the test execution result to obtain a test analysis result. According to the invention, the technical problems of low test efficiency and high test cost of a vehicle voice interaction system test method provided in the related art are solved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of smart cockpits, and in particular to a vehicle voice interaction system testing method, device, electronic equipment and storage medium. Background Art

[0002] As the core interactive component in the smart cockpit, the performance of the vehicle voice interaction system directly affects the user's driving experience and safety. The vehicle voice interaction system testing method in the related art usually adopts functional testing or model testing to implement the vehicle voice interaction system testing. Among them, the functional test focuses on verifying the execution accuracy of a single function to ensure that each function can accurately respond to user voice commands, while the model test focuses on evaluating the performance of the algorithm, such as recall rate and accuracy, to optimize the algorithm's understanding and processing of user voice commands. However, the two testing methods in the related art have the problems of low testing efficiency and high testing cost, which makes it difficult to achieve large-scale vehicle voice interaction system testing.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present disclosure provide a vehicle voice interaction system testing method, device, electronic device and storage medium to at least solve the technical problems of low testing efficiency and high testing cost in the vehicle voice interaction system testing method provided in the related art.

[0005] According to one aspect of an embodiment of the present disclosure, a vehicle voice interaction system testing method is provided, including: obtaining a target benchmark dictionary, wherein the target benchmark dictionary is used to represent a mapping relationship between a functional test case and a vehicle test log, the functional test case is used to verify whether at least one voice interaction function is executed according to the expected functional result, and the vehicle test log is used to determine a data flow record of a voice interaction link; performing mapping processing using preset intent labels and a target benchmark dictionary to obtain a target mapping result, wherein the target mapping result is used to determine a mapping relationship between a functional test case and an algorithm test case, and the algorithm test case is used to verify whether a voice interaction algorithm is executed according to the expected algorithm result; performing test processing on the vehicle voice interaction system based on the target mapping result to obtain a test execution result; and performing analysis processing on the test execution result to obtain a test analysis result.

[0006] Optionally, obtaining the target benchmark dictionary includes: obtaining functional test cases, wherein the functional test cases include: user voice commands, vehicle preconditions, expected execution results and functional description information; performing end-to-end functional testing based on the functional test cases to obtain a vehicle test log; parsing the vehicle test log to obtain a log parsing result, wherein the log parsing result is used to extract data flow records of the voice interaction link from the vehicle test log; and constructing a target benchmark dictionary based on the functional test cases and the log parsing results.

[0007] Optionally, mapping processing is performed using preset intent labels and a target reference dictionary to obtain a target mapping result including: determining a first mapping relationship based on the preset intent label and the algorithm test case, wherein the first mapping relationship is used to represent the mapping relationship between the first function point and the first voice instruction corresponding to the algorithm test case and the preset intent label; determining a second mapping relationship corresponding to the functional test case based on the target reference dictionary, wherein the second mapping relationship is used to represent the mapping relationship between the second function point and the second voice instruction corresponding to the functional test case and the preset intent label; and determining the target mapping result using the first mapping relationship and the second mapping relationship.

[0008] Optionally, test processing is performed on the vehicle voice interaction system based on the target mapping result to obtain the test execution result, including: converting the algorithm test case based on the target mapping result to obtain the target conversion result; and using the target conversion result to perform test processing on the vehicle voice interaction system to obtain the test execution result.

[0009] Optionally, the algorithm test case is converted based on the target mapping result to obtain the target conversion result, including: using the target mapping result and the target benchmark dictionary to determine the test expected data set corresponding to the algorithm test case; and determining the target conversion result based on the test expected data set and the algorithm test case.

[0010] Optionally, the target conversion result is used to perform test processing on the vehicle voice interaction system to obtain the test execution result, including: using the target test tool to perform distributed execution on the target conversion result to obtain the test execution result, wherein the target test tool is used to represent the vehicle-side open interface corresponding to the vehicle voice interaction system.

[0011] Optionally, analyzing and processing the test execution result to obtain the test analysis result includes: comparing the test execution result with the expected test result to obtain the test comparison result; and analyzing and processing the test comparison result to obtain the test analysis result.

[0012] Optionally, the vehicle voice interaction system testing method also includes: uploading the target benchmark dictionary, functional test cases and algorithm test cases to the cloud testing platform, so that the cloud testing platform performs test processing on the vehicle voice interaction system to obtain test execution results.

[0013] According to one embodiment of the present disclosure, a vehicle voice interaction system testing device is also provided, including: an acquisition module, used to acquire a target benchmark dictionary, wherein the target benchmark dictionary is used to represent the mapping relationship between functional test cases and vehicle test logs, the functional test cases are used to verify whether at least one voice interaction function is executed according to the expected functional results, and the vehicle test logs are used to determine the data flow records of the voice interaction link; a mapping module, used to perform mapping processing using preset intent labels and the target benchmark dictionary to obtain a target mapping result, wherein the target mapping result is used to determine the mapping relationship between the functional test cases and the algorithm test cases, and the algorithm test cases are used to verify whether the voice interaction algorithm is executed according to the expected algorithm results; a testing module, used to perform test processing on the vehicle voice interaction system based on the target mapping result to obtain a test execution result; an analysis module, used to perform analysis processing on the test execution result to obtain a test analysis result.

[0014] Optionally, the acquisition module is also used to: obtain functional test cases, wherein the functional test cases include: user voice commands, vehicle preconditions, expected execution results and functional description information; perform end-to-end functional testing based on the functional test cases to obtain vehicle test logs; parse the vehicle test logs to obtain log parsing results, wherein the log parsing results are used to extract data flow records of the voice interaction link from the vehicle test logs; and construct a target benchmark dictionary based on the functional test cases and the log parsing results.

[0015] Optionally, the mapping module is also used to: determine a first mapping relationship based on a preset intent label and an algorithm test case, wherein the first mapping relationship is used to represent the mapping relationship between a first function point corresponding to the algorithm test case, a first voice instruction and a preset intent label; determine a second mapping relationship corresponding to the functional test case based on a target reference dictionary, wherein the second mapping relationship is used to represent the mapping relationship between a second function point corresponding to the functional test case, a second voice instruction and a preset intent label; and determine a target mapping result using the first mapping relationship and the second mapping relationship.

[0016] Optionally, the test module is also used to: convert the algorithm test case based on the target mapping result to obtain the target conversion result; use the target conversion result to perform test processing on the vehicle voice interaction system to obtain the test execution result.

[0017] Optionally, the test module is also used to: determine the expected test data set corresponding to the algorithm test case using the target mapping result and the target reference dictionary; and determine the target conversion result based on the expected test data set and the algorithm test case.

[0018] Optionally, the test module is also used to: use a target test tool to perform distributed execution on the target conversion result to obtain a test execution result, wherein the target test tool is used to represent an open interface on the vehicle side corresponding to the vehicle voice interaction system.

[0019] Optionally, the analysis module is further used to: compare the test execution result with the expected test result to obtain a test comparison result; and analyze the test comparison result to obtain a test analysis result.

[0020] According to one embodiment of the present disclosure, an electronic device is also provided, including: a processor; a memory for storing processor executable instructions; wherein the processor is configured to execute instructions to implement the vehicle voice interaction system testing method in the embodiment of the present disclosure.

[0021] According to one embodiment of the present disclosure, a computer-readable storage medium is also provided, which includes a stored executable program, wherein when the executable program is running, the device where the storage medium is located is controlled to execute the vehicle voice interaction system testing method in the embodiment of the present disclosure.

[0022] According to one embodiment of the present disclosure, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the vehicle voice interaction test system method in the embodiment of the present disclosure is implemented.

[0023] In the disclosed embodiment, a target reference dictionary is obtained, and mapping processing is performed using preset intent labels and the target reference dictionary to obtain a target mapping result, and then a test processing is performed on the vehicle voice interaction system based on the target mapping result to obtain a test execution result, and finally the test execution result is analyzed and processed to obtain a test analysis result, thereby achieving the purpose of optimizing the performance of the vehicle voice interaction system, thereby achieving the technical effect of improving test efficiency and reducing test costs, and thereby solving the technical problems of low test efficiency and high test cost in the vehicle voice interaction system test method provided in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0025] Figure 1is a flow chart of a vehicle voice interaction test system method according to one embodiment of the present disclosure;

[0026] Figure 2 is a schematic diagram of a vehicle voice interaction system testing method according to one embodiment of the present disclosure;

[0027] Figure 3 It is a structural block diagram of a vehicle voice interaction test system device according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0030] The vehicle voice interaction system testing method in the related art usually adopts functional testing or model testing to implement the vehicle voice interaction system testing. Among them, functional testing focuses on verifying the execution accuracy of a single function to ensure that each function can accurately respond to user voice commands, while model testing focuses on evaluating the performance of the algorithm, such as recall rate and accuracy, to optimize the algorithm's understanding and processing of user voice commands. However, the two testing methods in the related art have the problems of low testing efficiency and high testing cost, which makes it difficult to implement large-scale vehicle voice interaction system testing.

[0031] Specifically, model testing can evaluate the recall and accuracy of the algorithm, but this testing method often cannot go deep into the complete process from the user issuing a command to the actual function execution. This limitation leads to blind spots in the voice interaction link that are not fully detected during batch testing, which may have an adverse impact on the final product performance and user experience. On the other hand, although functional testing can cover the entire path of voice execution and ensure the comprehensiveness of the test, this method usually requires manual intervention, which not only reduces the efficiency of the test, but also increases the cost of the test. In the modern fast-paced product development cycle, the above two testing methods with low testing efficiency and high testing costs are difficult to meet the market's demand for rapid iteration and high-quality output.

[0032] According to an embodiment of the present disclosure, a method embodiment of a vehicle voice interaction system testing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0033] The method embodiment can be executed in an electronic device or a similar computing device including a memory and a processor. Taking running on a computer terminal as an example, the computer terminal may include one or more processors (processors may include but are not limited to central processing units (CPU), graphics processing units (GPU), digital signal processing (DSP) chips, microprocessors (MCU), programmable logic devices (Field Programmable Gate Array, FPGA), neural network processors (Neural-network Processor Unit, NPU), tensor processors (TensorProcessing Unit, TPU), artificial intelligence (Artificial Intelligence, AI) type processors and other processing devices) and a memory for storing data. Optionally, the above-mentioned computer terminal may also include a transmission device, an input and output device, and a display device for communication functions. It can be understood by those of ordinary skill in the art that the above-mentioned structural description is only for illustration, and it does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include more or fewer components than the above-mentioned structural description, or have a configuration different from the above-mentioned structural description.

[0034] The memory can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the vehicle voice interaction system test method in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the computer program stored in the memory, that is, the above-mentioned vehicle voice interaction system test method is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0036] The display device may be, for example, a touch screen type liquid crystal display (LCD) and a touch display (also referred to as a "touch screen" or "touch display screen"). The liquid crystal display may enable a user to interact with a user interface of a mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), and a user may interact with the GUI by finger contacts and / or gestures on a touch-sensitive surface, wherein the human-computer interaction functions here may optionally include the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital videos, playing digital music and / or web browsing, etc. The executable instructions for executing the above human-computer interaction functions are configured / stored in a computer program product or readable storage medium executable by one or more processors.

[0037] Figure 1 is a flow chart of a vehicle voice interaction system testing method according to one embodiment of the present disclosure, such as Figure 1 As shown, the method comprises the following steps:

[0038] Step S11, obtaining a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a functional test case and a vehicle test log, the functional test case is used to verify whether at least one voice interaction function is executed according to the expected result of the function, and the vehicle test log is used to determine a data flow record of a voice interaction link;

[0039] Step S12, mapping processing is performed using the preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine the mapping relationship between the functional test case and the algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to the expected result of the algorithm;

[0040] Step S13, performing a test process on the vehicle voice interaction system based on the target mapping result to obtain a test execution result;

[0041] Step S14, analyzing and processing the test execution result to obtain a test analysis result.

[0042] The target benchmark dictionary is a detailed database or data set that records and standardizes the key data points and expected results of the entire process of voice assistants from user voice command parsing, understanding, execution to feedback, and is used to represent the mapping relationship between functional test cases and vehicle test logs. The generation process of the target benchmark dictionary includes three parts: case collection, real vehicle recording, and log parsing.

[0043] The above functional test cases are used to verify whether the specific functions of the software system are running normally according to the specifications or user requirements. They mainly focus on the user's voice command input and the vehicle's actual response and voice feedback output. It is a typical black box testing method. The purpose of the functional test case is to verify the integrity of the voice interaction link and the correctness of the function.

[0044] For example, when conducting a real vehicle test, for each specific function point, only one representative voice command needs to be verified to check whether the entire interactive link from user voice input to vehicle-side function execution is smooth, and whether the final execution result is consistent with expectations. For example, for the vehicle control function, the command "open the window" can be issued to determine whether the vehicle has executed the action of "opening the window" and whether it has replied with the voice feedback content of "OK, I opened it for you" to verify whether the system can correctly understand the command and execute the corresponding control function, while ensuring that the text-to-speech (TTS) reply output is accurate and the instructions are clear; for the navigation function, the command "go to the nearest gas station" can be issued to verify whether the system can correctly understand the command, call the navigation algorithm, plan the route, and finally issue the correct navigation execution signal, while ensuring that the TTS reply output is accurate and the instructions are clear. In this way, not only can potential problems in the voice interaction link be detected, but the effectiveness and comprehensiveness of the test can also be ensured under the smallest test set, thereby reducing the consumption of test resources and improving overall test efficiency while ensuring the test quality.

[0045] The above-mentioned vehicle test log is the real vehicle log, which refers to the collection of data and information recorded by the vehicle's electronic control system, software system or specialized test equipment when testing vehicle systems or components. The vehicle test log not only records the process of receiving, understanding and executing voice commands, but also includes the vehicle's actual response to the above commands, as well as the interaction between the voice assistant and other vehicle systems. By analyzing the vehicle test log, the functional integrity, performance, user experience, and integration of the voice assistant with the vehicle system can be evaluated, thereby identifying and resolving potential problems.

[0046] The above data flow record is to parse the real vehicle log. By performing a detailed analysis on the log generated when executing functional test cases in the real vehicle environment, the dynamic behavior of the voice interaction function in the entire process from user voice command input to vehicle-side function execution is captured and recorded.

[0047] The above-mentioned preset intent label is to define the intent of the semantic understanding model as a unified label for mapping the relationship between function points. In the test process, the role of the preset intent label is to map the user's voice command to its expected function point, so that the relevant test cases can be automatically executed.

[0048] For example, in the voice assistant test of the smart cockpit, the preset intent tags may include "navigate to...", "adjust the air conditioning temperature", "play music", etc. Each tag corresponds to one or more specific voice commands and expected system responses. By matching the instructions in the algorithm test case with its preset intent tags, it can be ensured that the test covers the entire end-to-end process of the voice assistant from receiving instructions to executing functions, thereby verifying the accuracy and reliability of the voice assistant.

[0049] The above-mentioned mapping process includes two parts: label definition and mapping rules, wherein label refers to the smallest unit to distinguish each functional point; label definition refers to using the intention label of the semantic understanding model as the benchmark for mapping the functional point relationship, so that all test cases are classified and identified in the same way based on the label, and are used to map the user voice instructions and their expected input and output of the benchmark dictionary to the user statements under the same functional point of the algorithm test set; mapping rules refer to mapping algorithm test cases with functional test cases based on unified labels, ensuring that each algorithm test case can be mapped to a functional test case and its corresponding vehicle preconditions and test execution results.

[0050] The above algorithm test cases usually refer to a set of test inputs and expected outputs designed to verify the correctness and performance of the algorithm. They mainly focus on the performance and accuracy of core algorithms such as speech recognition, semantic understanding, and dialogue management. They are a type of white box testing. Algorithm test cases include but are not limited to algorithm inputs, expected algorithm outputs, and evaluation indicators. Unlike functional test cases, algorithm test cases focus more on verifying the performance of the algorithm under ideal or controlled conditions, without involving the interaction of actual vehicle hardware or user interfaces.

[0051] For example, the intent understood by the Natural Language Understanding (NLU) model is tested accurately through the Application Programming Interface (API). For example, for "open the car window", it is determined whether the NLU result is mapped to the algorithm model output action = WindowsOpen, and the algorithm model output slot = {"set_type":"open again", "device":"car window", "value":"a little"}. By comparing the intent recognition result output by the model with the preset correct answer, it is tested whether the understood intent is accurate, so as to effectively identify and locate the possible problems of the model in understanding user instructions, thereby ensuring the functional accuracy and user experience of the voice assistant in the smart cockpit environment.

[0052] The above-mentioned vehicle voice interaction system is the intelligent cockpit voice assistant that performs batch testing. The intelligent cockpit voice assistant is a technology integrated in new energy vehicles that provides a convenient interactive experience by understanding and executing the user's voice commands. The intelligent cockpit voice assistant uses advanced voice recognition and semantic understanding technology, combined with vehicle control logic, to achieve voice control of vehicle functions such as navigation, music playback, vehicle settings adjustment, etc., thereby enhancing the user experience and making the driving process more intelligent and personalized.

[0053] Based on the above steps S11 to S14, by obtaining a target reference dictionary; performing mapping processing using preset intent labels and the target reference dictionary to obtain a target mapping result; performing test processing on the vehicle voice interaction system based on the target mapping result to obtain a test execution result; and analyzing and processing the test execution result to obtain a test analysis result, the purpose of optimizing the performance of the vehicle voice interaction system is achieved, thereby achieving the technical effect of improving test efficiency and reducing test costs, and further solving the technical problems of low test efficiency and high test cost in the vehicle voice interaction system test method provided in the related art.

[0054] The vehicle voice interaction system testing method in the embodiment of the present disclosure is further introduced below.

[0055] Optionally, in step S11, obtaining a target reference dictionary includes:

[0056] Step S111, obtaining a functional test case, wherein the functional test case includes: user voice command, vehicle precondition, expected execution result and functional description information;

[0057] Step S112, performing end-to-end functional testing based on the functional test case to obtain a vehicle test log;

[0058] Step S113, parsing the vehicle test log to obtain a log parsing result, wherein the log parsing result is used to extract a data flow record of a voice interaction link from the vehicle test log;

[0059] Step S114, constructing a target reference dictionary according to the functional test cases and the log parsing results.

[0060] The above functional test cases include: user voice commands, vehicle preconditions, expected execution results and functional description information. Among them, user voice commands refer to the voice commands issued by users to the vehicle intelligent system through natural language or specific commands. Voice commands are the starting point for users to interact with the vehicle intelligent system and are used to convey user needs; vehicle preconditions, namely vehicle-computer perception point inputs, refer to a series of conditions that the vehicle intelligent system needs to meet before executing user voice commands to ensure that the commands are executed in an appropriate environment to avoid potential risks or errors; expected execution results refer to the specific actions or states that the vehicle intelligent system should perform or reach after the user issues a voice command; functional description information is a detailed description of a specific function of the vehicle intelligent system, which can help testers understand the system design intent and ensure that the test cases can fully cover various possible usage scenarios, thereby verifying the integrity and reliability of the function.

[0061] Table 1 is a functional test case according to one embodiment of the present disclosure. As shown in Table 1, the functional test case includes a client case and a unified label. The client case includes a case title, test premise, test steps, and expected results.

[0062] For example, taking the use case title "Outside temperature is greater than 30 degrees - turn on rapid cooling" as an example, the test premise is "1. The network is connected, 2. In any interface, 3. Panoramic voice is turned on, 4. The outside temperature is greater than 30 degrees", and the test steps are to issue a voice command of "air conditioning maximum", and the expected result is to receive the voice feedback of "rapid cooling is turned on" and realize the function of "turn on rapid cooling", and define the unified label of the above client use case as "ac_hvac_max", indicating that this user instruction is intended to require the vehicle air-conditioning system to reach the maximum cooling capacity, that is, to turn on the rapid cooling mode.

[0063] For example, user voice commands include but are not limited to specific commands such as "turn on the air conditioner" or "navigate to the nearest gas station"; vehicle preconditions include but are not limited to the current operating status of the vehicle, such as whether the engine is started, whether the vehicle is stationary or moving, external environmental information, such as weather conditions, road conditions, and personalized system settings, such as user preferences, historical operation records, etc. Taking the user voice command "seat recline" as an example, before issuing the execution action, the current vehicle status will be judged first, and the configuration information of the vehicle supporting the "seat recline" function will be obtained. If the vehicle supports this function, the action of executing the seat recline and the corresponding reply will be output. If not, the corresponding action will not be executed and the reply "this function is not supported" will be given.

[0064] Table 1 Functional test cases

[0065]

[0066] The above end-to-end functional testing is performed in the real car, and the preconditions and execution process of the use case are recorded through the log of the real car. The end-to-end functional testing method is different from the traditional functional testing or model testing. It focuses on the testing of the entire chain from the user issuing the voice command to the vehicle system finally executing the command, ensuring the coherence and correctness of the links such as voice recognition, semantic understanding, function calling and execution feedback.

[0067] Exemplarily, before conducting a test, vehicle preconditions are first set, such as vehicle position, vehicle speed, in-vehicle temperature, entertainment system status, etc., and the vehicle preconditions are used as the background environment for executing voice commands; secondly, voice commands are issued to the voice assistant in the vehicle, such as "navigate to the airport", "adjust the air-conditioning temperature to 22 degrees", etc.; finally, the vehicle system records the user's voice commands, vehicle preconditions, expected execution results, and function description information to obtain a vehicle test log.

[0068] Table 2(a) is a vehicle log parsing result according to one embodiment of the present disclosure. As shown in Table 2(a), the vehicle log parsing result includes a voice command, a voice zone where the voice command is located, a vehicle terminal perception point input, and an algorithm model output action. When obtaining a target reference dictionary, it is necessary to define a unified label corresponding to the vehicle log parsing result.

[0069] Table 2 (a) Vehicle log analysis results

[0070]

[0071] Table 2(b) is a vehicle log parsing result according to one embodiment of the present disclosure. As shown in Table 2(b), the vehicle log parsing result includes the algorithm model output slot, the vehicle-side execution command output, and the vehicle-side voice reply output. When obtaining the target reference dictionary, it is necessary to define a unified label corresponding to the vehicle log parsing result.

[0072] Table 2(b) Vehicle log analysis results

[0073]

[0074] For example, taking the use case title "The temperature outside the car is greater than 30 degrees - turn on rapid cooling" as an example, a voice command of "air conditioning at maximum" is issued. The voice assistant executes relevant test steps according to the received voice command and parses the current real vehicle log to obtain the following results: the sound zone where the voice command is located is the low-frequency sound zone, the perception point input on the vehicle side is "{"ac.outside.temp":{"value":31}}", the algorithm model output action is "AcSet", the algorithm model output slot is "{"set_type":"Adjust to","device":"Air conditioning","value":"Maximum"}", the vehicle side execution command output is "['ac.fast.warming.on']", and the vehicle side voice reply output is "Rapid cooling is turned on".

[0075] Specifically, to achieve batch end-to-end voice interaction testing, we first need to obtain a series of functional test cases from the voice assistant function test of the smart cockpit. The functional test cases include the voice commands issued by the user, such as "turn on the air conditioner and set the temperature to 22 degrees", the vehicle preconditions, such as the vehicle is in idling state and the air conditioning system is available, and the expected execution results and description information of the function, such as the air conditioner is adjusted to 22 degrees according to the command. Subsequently, based on the above functional test cases, end-to-end functional testing is performed in the real vehicle environment of new energy vehicles. During the test, the vehicle's voice interaction system will respond to user commands and execute corresponding functions, while recording the entire interaction process, including voice recognition, semantic parsing, function execution feedback, etc., to form a detailed vehicle test log. After obtaining the real vehicle test log, it is parsed and processed to extract key information from it to form a log parsing result. The log parsing result includes the input of the vehicle perception point, that is, the vehicle preconditions, the input of the user's voice command, the output of the semantic understanding model, the output of the vehicle execution signal, and the TTS reply output. Finally, the target benchmark dictionary is constructed based on the functional test cases and the log parsing results. The benchmark dictionary will contain detailed information on the above test cases and data flow records obtained in the actual vehicle test as the basis for subsequent batch end-to-end testing. For example, for the instruction "turn on the air conditioner and set the temperature to 22 degrees", the benchmark dictionary will record how the semantic understanding model correctly parses the instruction under specific vehicle preconditions, how the vehicle system responds and executes, and how the voice assistant feeds back to the user. By building a benchmark dictionary during the batch testing process, it can ensure that the algorithm test cases can be accurately converted into end-to-end test cases, covering the entire process from user instructions to function execution, thereby improving testing efficiency.

[0076] Based on the above steps S111 to S114, by obtaining functional test cases and performing end-to-end functional tests based on the functional test cases, a vehicle test log is obtained, and then the vehicle test log is parsed to obtain a log parsing result. Finally, a target benchmark dictionary is constructed based on the functional test cases and the log parsing results. This can systematically record and standardize the correspondence between voice commands and function executions, thereby improving test efficiency.

[0077] Optionally, in step S12, mapping is performed using a preset intent tag and a target reference dictionary to obtain a target mapping result including:

[0078] Step S121, determining a first mapping relationship based on a preset intent label and an algorithm test case, wherein the first mapping relationship is used to represent a mapping relationship between a first function point corresponding to the algorithm test case, a first voice instruction, and a preset intent label;

[0079] Step S122, determining a second mapping relationship corresponding to the functional test case based on the target reference dictionary, wherein the second mapping relationship is used to represent a mapping relationship between a second function point corresponding to the functional test case, a second voice instruction, and a preset intention label;

[0080] Step S123: determine a target mapping result using the first mapping relationship and the second mapping relationship.

[0081] The first mapping relationship mentioned above refers to the mapping relationship between the algorithm test set and the preset intent label, which is used to represent the mapping relationship between the first function point and the first voice command corresponding to the algorithm test case and the preset intent label. The main goal is to map the intent, voice command and expected output in the algorithm test case to a unified label system to ensure that in batch testing, the algorithm-level test can be accurately connected with the actual function execution test.

[0082] The second mapping relationship mentioned above refers to the mapping relationship between the client test case and the preset intent label, which is used to represent the mapping relationship between the second function point and the second voice command corresponding to the algorithm test case and the preset intent label. The user voice commands, vehicle preconditions and expected results obtained through real vehicle testing are used to map the functional test case with the preset intent labels in the algorithm test case to ensure that the functional test case can be accurately converted into the form required by the algorithm test case for end-to-end batch testing.

[0083] Exemplarily, the algorithm test set includes the first function point and the first voice command. Assuming that the first function point is "air conditioning at maximum", the first voice command is "air conditioning at maximum", "air conditioning at maximum", "air conditioning at maximum", "air conditioning at maximum", "air conditioning at maximum", "air conditioning at maximum", and "air conditioning at maximum", etc., and setting their unified labels to "ac_hvac_max" to construct first mapping relationships such as "air conditioning at maximum - air conditioning at maximum - ac_hvac_max". The client use case includes the second function point and the second voice command. Assuming that the second function point is "air conditioning at maximum", the second voice command is "air conditioning at maximum", and setting their unified labels to "ac_hvac_max" to construct the second mapping relationship of "air conditioning at maximum - air conditioning at maximum - ac_hvac_max".

[0084] Table 3 is a unified label mapping result according to one embodiment of the present disclosure. As shown in Table 3, the mapping relationship between algorithm test cases and functional test cases can be achieved through unified labels.

[0085] Table 3 Unified label mapping results

[0086]

[0087] For example, taking the client use case function point of "air conditioning maximum" as an example, the corresponding voice command can be "air conditioning turned to maximum", and its unified label is defined as "ac_hvac_max"; for the algorithm test set with the function point of "air conditioning maximum", the corresponding voice command can be "air conditioning turned to maximum", "turn the air conditioning to maximum", "adjust the air conditioning to maximum", etc., and its unified label is defined as "ac_hvac_max", so that the algorithm test case and the functional test case can be mapped based on the same unified relationship.

[0088] Based on the above steps S121 to S123, by determining the first mapping relationship based on the preset intent label and the algorithm test case, determining the second mapping relationship corresponding to the functional test case based on the target benchmark dictionary, and determining the target mapping result using the first mapping relationship and the second mapping relationship, the algorithm test case and the functional test case can be effectively associated to ensure that each algorithm test case can be accurately mapped to the corresponding functional test case and its preconditions and execution results, thereby ensuring the comprehensiveness of end-to-end testing, while reducing the need for actual vehicle testing, reducing labor costs, and improving testing efficiency.

[0089] Optionally, in step S13, a test process is performed on the vehicle voice interaction system based on the target mapping result, and the test execution result obtained includes:

[0090] Step S131, converting the algorithm test case based on the target mapping result to obtain a target conversion result;

[0091] Step S132, using the target conversion result to perform test processing on the vehicle voice interaction system to obtain a test execution result.

[0092] The above conversion process refers to converting algorithm test cases into end-to-end test cases. The core of this process is to expand the test from the isolated algorithm level to the test of the entire system process, ensuring that every link from user input to system response is accurately tested. Through label mapping, end-to-end expectations are added to the algorithm test cases. There are various voice commands under the same function, that is, user statements, so batch testing and verification can be performed.

[0093] The above target conversion result is the end-to-end test case. Through the construction of a benchmark dictionary and the mapping of unified labels, the originally scattered algorithm test cases and functional test cases can be integrated to form a complete set of test cases for the entire process from the voice assistant receiving user voice commands to the execution of functions.

[0094] Table 4(a) is a batch test case according to one embodiment of the present disclosure. As shown in Table 4(a), the batch test case includes three parts: algorithm test set, unified label, and end-to-end expectation added after mapping. The end-to-end expectation added after mapping includes: case title, test premise, expected result and the audio zone.

[0095] Table 4(b) is a batch test case according to one embodiment of the present disclosure. As shown in Table 4(b), the batch test case includes three parts: algorithm test set, unified label, and end-to-end expectation added after mapping. Among them, the end-to-end expectation added after mapping includes: perception point input on the vehicle side, expected action of the algorithm model, expected slot of the algorithm model, expected execution command on the vehicle side, and expected voice response on the vehicle side.

[0096] Table 4 (a) Batch test cases

[0097]

[0098] Table 4(b) Batch test cases

[0099]

[0100] For example, taking the voice command "turn the air conditioner to the maximum" as an example, its unified label is "ac_hvac_max". Through unified label mapping, its mapping is obtained and end-to-end expectations are added. The end-to-end expected results include: the use case title is "The temperature outside the car is greater than 30 degrees-turn on rapid cooling", the test premise is "1. The network is connected; 2. In any interface; 3. Full-scene voice is turned on; 4. The temperature outside the car is greater than 30 degrees", the expected result is "1. Voice feedback: Rapid cooling is turned on; 2. Function: Turn on rapid cooling", the voice command is in the audio The area is "low-frequency sound area", the input of the perception point on the vehicle side is "{"ac.outside.temp":{"value":31}}", the expected action of the algorithm model output is "AcSet", the expected slot of the algorithm model output is "{"set_type":"adjust to","device":"air conditioning","value":"maximum"}", the expected execution command on the vehicle side is "['ac.fast.warming.on']", and the expected voice reply on the vehicle side is "fast cooling is on". According to the batch test cases, the recorded cases can be regressed through the end-to-end test tool without the need to verify again on the actual vehicle, thereby reducing the manpower cost of actual vehicle testing. In addition, since the simple model test only verifies the "brain" of the voice assistant, and the test conclusion cannot represent the real user experience, its action execution requires the cooperation of all links in the voice link, while the batch test is no longer limited to the NLU model test, and the conclusion of the end-to-end test can verify all links, so the test conclusion can better represent the actual vehicle user experience and is more credible.

[0101] Specifically, the intent of the algorithm test case is matched with the functional points in the benchmark dictionary, and the mapping is achieved by defining unified labels. Each algorithm test case not only contains its expected input and output at the algorithm level, but also associates the preconditions and expected execution results of the actual vehicle test, becoming a complete end-to-end test case. Finally, use an automated testing tool to perform the test. Ensure that the test tool can accept user voice commands and vehicle perception points as input, simulate the response process of the voice assistant in the actual use environment, collect and analyze the test results, and compare them with the expected output to evaluate the performance and reliability of the voice assistant.

[0102] Based on the above steps S131 to S132, the algorithm test cases are converted based on the target mapping results to obtain target conversion results, and the target conversion results are used to perform test processing on the vehicle voice interaction system to obtain test execution results. This can quickly cover a large number of test scenarios, reduce dependence on real vehicle resources, and reduce testing costs.

[0103] Optionally, in step S131, the algorithm test case is converted based on the target mapping result to obtain the target conversion result, including: using the target mapping result and the target benchmark dictionary to determine the expected test data set corresponding to the algorithm test case; and determining the target conversion result based on the expected test data set and the algorithm test case.

[0104] The above-mentioned expected test data set refers to the data set that contains the expected results of the end-to-end test after conversion and mapping. The expected test data set includes but is not limited to user voice commands, vehicle perception point input, output of the semantic understanding model, vehicle computer execution signals, TTS replies, etc.

[0105] Exemplarily, in combination with the target mapping result and the target benchmark dictionary, for each algorithm test case, its expected input and expected output in an end-to-end test environment are determined, thereby forming a test expected data set, and based on the test expected data set, the algorithm test case is converted into a form suitable for the end-to-end test environment to ensure that the algorithm test case can be correctly interpreted and executed, thereby determining the target conversion result based on the algorithm test case of the test expected data set.

[0106] Based on the above optional embodiments, by utilizing the target mapping results and the target benchmark dictionary to determine the test expected data set corresponding to the algorithm test case, and determining the target conversion result based on the test expected data set and the algorithm test case, it is possible to ensure that the algorithm test case fully covers the functional links of the voice assistant, thereby achieving the purpose of batch testing.

[0107] Optionally, in step S132, the target conversion result is used to perform test processing on the vehicle voice interaction system, and obtaining the test execution result includes: using the target test tool to perform distributed execution on the target conversion result to obtain the test execution result, wherein the target test tool is used to represent the vehicle-side open interface corresponding to the vehicle voice interaction system.

[0108] The above-mentioned target test tools are automated test tools. Automated test tools include but are not limited to the vehicle-to-everything application programming interface (V2X API), speech recognition engine, NLP module, speech synthesis technology, virtual vehicle environment simulator, and test result analysis and report generation system. Automated test tools work together to automatically execute the entire test process from user voice command input to function execution result output, while collecting and analyzing test data to evaluate the performance and stability of voice assistants in smart cockpits.

[0109] The above-mentioned distributed execution is batch execution of the distributed test architecture. By using the automated test tool to batch execute the test set, ensure that the test tool supports input parameters such as user voice commands and vehicle perception point input, and collect the execution results.

[0110] Exemplarily, first, the target test tool is used to communicate with the API interface corresponding to the vehicle voice interaction system; secondly, based on the benchmark dictionary and unified label mapping mechanism, the batch end-to-end test cases are converted into a format that can be understood and executed by the target test tool; finally, the V2X API is used to perform distributed execution of the converted test cases, and data from the entire process is collected, including the output of the semantic understanding model, the signal executed by the vehicle computer, and the TTS response.

[0111] Based on the above optional embodiment, the target conversion result is distributedly executed using a target test tool to obtain a test execution result, and testing can be performed simultaneously based on multiple nodes, thereby improving test efficiency and shortening the test cycle.

[0112] Optionally, in step S14, the test execution result is analyzed and processed, and the test analysis result obtained includes:

[0113] Step S141, comparing the test execution result with the test expected result to obtain a test comparison result;

[0114] Step S142, analyzing and processing the test comparison result to obtain a test analysis result.

[0115] Specifically, the test comparison results are obtained by comparing the test execution results with the expected test results of each link of the voice link. Based on the test comparison results, identify which test cases have execution results that do not meet expectations, and analyze whether the deviations come from specific links such as semantic understanding errors, abnormal function execution signals, and inaccurate TTS responses to determine the root cause of the problem. For the identified problems, further analysis is conducted to determine whether they are due to incomplete test case design, limitations of the algorithm model, or execution errors, and the impact of the test results on the performance of the voice assistant is evaluated, including key indicators such as recall rate, accuracy, or user experience.

[0116] Based on the above steps S141 to S142, the test execution results are compared with the expected test results to obtain test comparison results, and the test comparison results are analyzed and processed to obtain test analysis results. This can efficiently identify problems encountered during the test process and continuously optimize the performance of the voice assistant based on the identified problems, thereby improving the user experience in the smart cockpit.

[0117] Optionally, the vehicle voice interaction system testing method also includes: uploading the target benchmark dictionary, functional test cases and algorithm test cases to the cloud testing platform, so that the cloud testing platform performs test processing on the vehicle voice interaction system to obtain test execution results.

[0118] The above-mentioned cloud testing platform refers to an online service platform that uses cloud computing technology to test software or hardware. The cloud testing platform provides the virtual environment, testing tools, automated script execution, test data management, result analysis, and report generation required for testing. Users can access and use the above services through the Internet without having to build a local testing environment.

[0119] For example, first, the cloud test platform reads the test cases from the benchmark dictionary stored in the cloud; second, the cloud test platform uses its centrally managed automated testing tools to distribute the test cases to each test node according to the predetermined execution strategy. After receiving the test cases, the test nodes start to execute test steps such as speech recognition, semantic understanding, and function call. The speech recognition module is responsible for converting user voice commands into text, the semantic understanding module parses the text and determines the intent, and the function call module executes the corresponding vehicle function according to the parsed intent. After the test is completed, each node will collect the execution results, including the accuracy of speech recognition, the correctness of semantic understanding, the reliability of function execution, and the quality of TTS responses, and feedback to the cloud test platform.

[0120] Based on the above optional embodiments, the target benchmark dictionary, functional test cases and algorithm test cases are uploaded to the cloud test platform, and the vehicle voice interaction system is tested based on the cloud test platform to obtain the test execution results, which can reduce the dependence on actual vehicle testing, reduce labor costs, and ensure the performance and user experience of the voice assistant in the smart cockpit.

[0121] Figure 2 is a schematic diagram of a vehicle voice interaction system testing method according to one embodiment of the present disclosure, such as Figure 2As shown in the figure, during the test process of the vehicle voice interaction system, firstly, a benchmark dictionary is constructed through case collection, real vehicle recording and log analysis; secondly, based on the unified label mapping relationship, function point-label definition is performed to obtain the function point-label relationship dictionary; thirdly, test case conversion is performed to confirm the function point label of the algorithm test case, and the vehicle preconditions, test execution results and expected additions under the unified label are queried according to the benchmark dictionary, so as to map them to the end-to-end batch test case; finally, batch end-to-end test execution is performed based on the end-to-end batch test case, and batch test execution is performed by configuring the end-to-end test tool to obtain the test result file, and the result analysis is performed according to the test result file to identify the problems in the function execution, so as to make optimization adjustments. By constructing a benchmark dictionary and unified label mapping, comprehensive end-to-end test coverage is achieved, and at the same time, through automated batch testing, the test efficiency is improved and the test cost of real vehicle testing is reduced.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present disclosure.

[0123] In the embodiments of the present disclosure, a vehicle voice interaction system testing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0124] Figure 3 is a structural block diagram of a vehicle voice interaction system testing device according to one embodiment of the present disclosure, such as Figure 3 As shown, the device comprises:

[0125] An acquisition module 301 is used to acquire a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a functional test case and a vehicle test log, the functional test case is used to verify whether at least one voice interaction function is executed according to the expected result of the function, and the vehicle test log is used to determine a data flow record of the voice interaction link;

[0126] A mapping module 302 is used to perform mapping processing using a preset intent label and a target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine a mapping relationship between a functional test case and an algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to the expected result of the algorithm;

[0127] A test module 303 is used to perform a test process on the vehicle voice interaction system based on the target mapping result to obtain a test execution result;

[0128] The analysis module 304 is used to analyze and process the test execution results to obtain test analysis results.

[0129] Optionally, the acquisition module 301 is also used to: obtain functional test cases, wherein the functional test cases include: user voice commands, vehicle preconditions, expected execution results and functional description information; perform end-to-end functional testing based on the functional test cases to obtain a vehicle test log; parse the vehicle test log to obtain a log parsing result, wherein the log parsing result is used to extract data flow records of the voice interaction link from the vehicle test log; and construct a target benchmark dictionary based on the functional test cases and the log parsing results.

[0130] Optionally, the mapping module 302 is also used to: determine a first mapping relationship based on a preset intent label and an algorithm test case, wherein the first mapping relationship is used to represent the mapping relationship between a first function point corresponding to the algorithm test case, a first voice instruction and a preset intent label; determine a second mapping relationship corresponding to the functional test case based on a target reference dictionary, wherein the second mapping relationship is used to represent the mapping relationship between a second function point corresponding to the functional test case, a second voice instruction and the preset intent label; and determine a target mapping result using the first mapping relationship and the second mapping relationship.

[0131] Optionally, the test module 303 is also used to: convert the algorithm test case based on the target mapping result to obtain a target conversion result; and perform test processing on the vehicle voice interaction system using the target conversion result to obtain a test execution result.

[0132] Optionally, the testing module 303 is further used to: determine the expected test data set corresponding to the algorithm test case using the target mapping result and the target reference dictionary; and determine the target conversion result based on the expected test data set and the algorithm test case.

[0133] Optionally, the test module is also used to: use a target test tool to perform distributed execution on the target conversion result to obtain a test execution result, wherein the target test tool is used to represent an open interface on the vehicle side corresponding to the vehicle voice interaction system.

[0134] Optionally, the analysis module 304 is further used to: compare the test execution result with the expected test result to obtain a test comparison result; and analyze the test comparison result to obtain a test analysis result.

[0135] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0136] According to one embodiment of the present disclosure, an electronic device is also provided, including: a processor; a memory for storing processor executable instructions; wherein the processor is configured to execute instructions to implement the vehicle voice interaction system testing method in the embodiment of the present disclosure.

[0137] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0138] Step S1, obtaining a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a functional test case and a vehicle test log, the functional test case is used to verify whether at least one voice interaction function is executed according to the expected result of the function, and the vehicle test log is used to determine a data flow record of a voice interaction link;

[0139] Step S2, mapping processing is performed using the preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine the mapping relationship between the functional test case and the algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to the expected result of the algorithm;

[0140] Step S3, performing a test process on the vehicle voice interaction system based on the target mapping result to obtain a test execution result;

[0141] Step S4, analyzing and processing the test execution result to obtain a test analysis result.

[0142] According to one embodiment of the present disclosure, a computer-readable storage medium is also provided, which includes a stored executable program, wherein when the executable program is running, the device where the storage medium is located is controlled to execute the vehicle voice interaction system testing method in the embodiment of the present disclosure.

[0143] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0144] Step S1, obtaining a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a functional test case and a vehicle test log, the functional test case is used to verify whether at least one voice interaction function is executed according to the expected result of the function, and the vehicle test log is used to determine a data flow record of a voice interaction link;

[0145] Step S2, mapping processing is performed using the preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine the mapping relationship between the functional test case and the algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to the expected result of the algorithm;

[0146] Step S3, performing a test process on the vehicle voice interaction system based on the target mapping result to obtain a test execution result;

[0147] Step S4, analyzing and processing the test execution result to obtain a test analysis result.

[0148] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0149] According to one embodiment of the present disclosure, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the vehicle voice interaction test system method in the embodiment of the present disclosure is implemented.

[0150] Optionally, in this embodiment, the computer program product may be configured as a computer program for executing the following steps:

[0151] Step S1, obtaining a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a functional test case and a vehicle test log, the functional test case is used to verify whether at least one voice interaction function is executed according to the expected result of the function, and the vehicle test log is used to determine a data flow record of a voice interaction link;

[0152] Step S2, mapping processing is performed using the preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine the mapping relationship between the functional test case and the algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to the expected result of the algorithm;

[0153] Step S3, performing a test process on the vehicle voice interaction system based on the target mapping result to obtain a test execution result;

[0154] Step S4, analyzing and processing the test execution result to obtain a test analysis result.

[0155] In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0156] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present disclosure. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk, etc. Various media that can store program codes.

[0160] The above are only preferred embodiments of the present disclosure. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present disclosure. These improvements and modifications should also be regarded as the protection scope of the present disclosure.

Claims

1. A vehicle voice interaction system testing method, characterized in that: include: Obtaining a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a functional test case and a vehicle test log, wherein the functional test case is used to verify whether at least one voice interaction function is executed according to the expected result of the function, and the vehicle test log is used to determine a data flow record of the voice interaction link; Mapping is performed using the preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine the mapping relationship between the functional test case and the algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to the expected result of the algorithm; Performing a test process on the vehicle voice interaction system based on the target mapping result to obtain a test execution result; The test execution result is analyzed and processed to obtain a test analysis result.

2. The method according to claim 1, characterized in that Acquiring the target reference dictionary includes: Acquire the functional test case, wherein the functional test case includes: user voice command, vehicle precondition, expected execution result and functional description information; Perform end-to-end functional testing based on the functional test case to obtain the vehicle test log; Parsing the vehicle test log to obtain a log parsing result, wherein the log parsing result is used to extract the data flow record of the voice interaction link from the vehicle test log; The target reference dictionary is constructed according to the functional test case and the log parsing result.

3. The method according to claim 1, characterized in that Using the preset intent label and the target reference dictionary to perform mapping processing to obtain the target mapping result includes: Determine a first mapping relationship based on the preset intent label and the algorithm test case, wherein the first mapping relationship is used to represent a mapping relationship between a first function point corresponding to the algorithm test case, a first voice instruction and the preset intent label; Determine a second mapping relationship corresponding to the functional test case based on the target reference dictionary, wherein the second mapping relationship is used to represent a mapping relationship between a second function point corresponding to the functional test case, a second voice instruction and the preset intention label; The target mapping result is determined using the first mapping relationship and the second mapping relationship.

4. The method according to claim 1, characterized in that: Performing a test process on the vehicle voice interaction system based on the target mapping result, and obtaining the test execution result includes: Based on the target mapping result, the algorithm test case is converted to obtain a target conversion result; The target conversion result is used to perform test processing on the vehicle voice interaction system to obtain the test execution result.

5. The method according to claim 4, characterized in that The algorithm test case is converted based on the target mapping result to obtain the target conversion result, including: Determine the test expected data set corresponding to the algorithm test case using the target mapping result and the target reference dictionary; The target conversion result is determined based on the expected test data set and the algorithm test case.

6. The method according to claim 4, characterized in that The vehicle voice interaction system is tested using the target conversion result, and the test execution result obtained includes: The target conversion result is distributedly executed using a target test tool to obtain the test execution result, wherein the target test tool is used to represent the vehicle-side open interface corresponding to the vehicle voice interaction system.

7. The method according to claim 1, characterized in that Analyzing and processing the test execution result to obtain the test analysis result includes: Comparing the test results with the expected test results to obtain a test comparison result. The test comparison result is analyzed and processed to obtain the test analysis result.

8. The method according to claim 1, characterized in that The method further comprises: The target reference dictionary, the functional test cases and the algorithm test cases are uploaded to a cloud test platform so that the cloud test platform performs test processing on the vehicle voice interaction system to obtain the test execution result.

9. A vehicle voice interaction system testing device, characterized in that: include: an acquisition module, configured to acquire a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a functional test case and a vehicle test log, wherein the functional test case is used to verify whether at least one voice interaction function is executed according to the expected result of the function, and the vehicle test log is used to determine a data flow record of a voice interaction link; A mapping module, used to perform mapping processing using a preset intent tag and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine a mapping relationship between the functional test case and the algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to the expected result of the algorithm; A test module, used to perform a test process on the vehicle voice interaction system based on the target mapping result to obtain a test execution result; The analysis module is used to analyze and process the test execution results to obtain test analysis results.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the vehicle voice interaction system testing method described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the storage medium is located is controlled to execute the vehicle voice interaction system testing method described in any one of claims 1 to 8.

12. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the vehicle voice interaction system testing method according to any one of claims 1 to 8.

Images