Vehicle voice interaction system test method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a vehicle voice interaction system test method and device, electronic equipment and storage medium. Wherein, the method comprises: obtaining a target reference dictionary; performing mapping processing using a preset intent label and the target reference dictionary to obtain a target mapping result; performing test processing on the vehicle voice interaction system based on the target mapping result to obtain a test execution result; and performing analysis processing on the test execution result to obtain a test analysis result. The present disclosure solves the technical problems of low test efficiency and high test cost of the vehicle voice interaction system test method provided in the related art.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of intelligent cockpit, in particular, to a vehicle voice interaction system testing method and device, electronic equipment and storage medium. BACKGROUND

[0002] As a core interaction component in the intelligent cockpit, the performance of the vehicle voice interaction system directly affects the driving experience and safety of the user. The vehicle voice interaction system testing method in the related art usually adopts function testing or model testing to implement vehicle voice interaction system testing. The function testing focuses on verifying the execution accuracy of a single function, ensuring that each function can accurately respond to the user voice instruction. The model testing focuses on evaluating the performance of the algorithm, such as recall rate and accuracy, to optimize the understanding and processing of the algorithm to the user voice instruction. However, the two testing methods in the related art have the problems of low testing efficiency and high testing cost, thereby it is difficult to implement large-scale vehicle voice interaction system testing.

[0003] At present, no effective solution has been proposed for the above problems. SUMMARY

[0004] The embodiments of the present disclosure provide a vehicle voice interaction system testing method, device, electronic equipment and storage medium, to at least solve the technical problems of low testing efficiency and high testing cost of the vehicle voice interaction system testing method provided in the related art.

[0005] According to an aspect of an embodiment of the present disclosure, a vehicle voice interaction system testing method is provided, comprising: obtaining a target reference dictionary, wherein the target reference dictionary is used to represent the mapping relationship between a function test case and a vehicle test log, the function test case is used to verify whether at least one voice interaction function is executed according to a function expected result, and the vehicle test log is used to determine the data flow record of a voice interaction link; performing mapping processing by using a preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine the mapping relationship between the function test case and an algorithm test case, and the algorithm test case is used to verify whether a voice interaction algorithm is executed according to an algorithm expected result; performing testing processing on the vehicle voice interaction system based on the target mapping result to obtain a testing execution result; and performing analysis processing on the testing execution result to obtain a testing analysis result.

[0006] Optionally, obtaining the target benchmark dictionary includes: obtaining functional test cases, wherein the functional test cases include: user voice commands, vehicle preconditions, expected execution results, and functional description information; performing end-to-end functional testing based on the functional test cases to obtain vehicle test logs; parsing the vehicle test logs to obtain log parsing results, wherein the log parsing results are used to extract data flow records of the voice interaction link from the vehicle test logs; and constructing the target benchmark dictionary based on the functional test cases and the log parsing results.

[0007] Optionally, the mapping process using preset intent tags and a target benchmark dictionary to obtain the target mapping result includes: determining a first mapping relationship based on preset intent tags and algorithm test cases, wherein the first mapping relationship is used to represent the mapping relationship between the first functional point and the first voice command corresponding to the algorithm test case and the preset intent tag; determining a second mapping relationship corresponding to the functional test case based on the target benchmark dictionary, wherein the second mapping relationship is used to represent the mapping relationship between the second functional point and the second voice command corresponding to the functional test case and the preset intent tag; and determining the target mapping result using the first mapping relationship and the second mapping relationship.

[0008] Optionally, the test processing of the vehicle voice interaction system based on the target mapping result to obtain the test execution result includes: transforming the algorithm test cases based on the target mapping result to obtain the target transformation result; and using the target transformation result to perform test processing on the vehicle voice interaction system to obtain the test execution result.

[0009] Optionally, the algorithm test cases are transformed based on the target mapping results to obtain the target transformation results, including: determining the expected test dataset corresponding to the algorithm test cases using the target mapping results and the target benchmark dictionary; and determining the target transformation results based on the expected test dataset and the algorithm test cases.

[0010] Optionally, the target conversion result is used to perform test processing on the vehicle voice interaction system to obtain test execution results, including: using a target testing tool to perform distributed execution on the target conversion result to obtain test execution results, wherein the target testing tool is used to represent the open interface of the vehicle terminal corresponding to the vehicle voice interaction system.

[0011] Optionally, the test execution results are analyzed and processed to obtain test analysis results, including: comparing the test execution results with the expected test results to obtain test comparison results; and analyzing and processing the test comparison results to obtain test analysis results.

[0012] Optionally, the vehicle voice interaction system testing method further comprises: uploading the target reference dictionary, the function test case and the algorithm test case to a cloud testing platform, so that the cloud testing platform performs testing processing on the vehicle voice interaction system to obtain a test execution result.

[0013] According to an embodiment of the present disclosure, a vehicle voice interaction system testing device is also provided, comprising: an acquisition module configured to acquire a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a function test case and a vehicle test log, the function test case is used to verify whether at least one voice interaction function is executed according to a function expected result, and the vehicle test log is used to determine a data flow record of a voice interaction link; a mapping module configured to perform mapping processing by using a preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine a mapping relationship between the function test case and an algorithm test case, and the algorithm test case is used to verify whether a voice interaction algorithm is executed according to an algorithm expected result; a testing module configured to perform testing processing on a vehicle voice interaction system based on the target mapping result to obtain a test execution result; and an analysis module configured to perform analysis processing on the test execution result to obtain a test analysis result.

[0014] Optionally, the acquisition module is further configured to: acquire the function test case, wherein the function test case comprises a user voice instruction, a vehicle precondition, an expected execution result and function description information; perform end-to-end function testing based on the function test case to obtain a vehicle test log; and perform analysis processing on the vehicle test log to obtain a log analysis result, wherein the log analysis result is used to extract a data flow record of a voice interaction link from the vehicle test log; and construct the target reference dictionary according to the function test case and the log analysis result.

[0015] Optionally, the mapping module is further configured to: determine a first mapping relationship based on the preset intent label and the algorithm test case, wherein the first mapping relationship is used to represent a mapping relationship between a first function point corresponding to the algorithm test case, a first voice instruction and the preset intent label; determine a second mapping relationship corresponding to the function test case based on the target reference dictionary, wherein the second mapping relationship is used to represent a mapping relationship between a second function point corresponding to the function test case, a second voice instruction and the preset intent label; and determine the target mapping result by using the first mapping relationship and the second mapping relationship.

[0016] Optionally, the testing module is further configured to: perform conversion processing on the algorithm test case based on the target mapping result to obtain a target conversion result; and perform testing processing on the vehicle voice interaction system by using the target conversion result to obtain the test execution result.

[0017] Optionally, the test module is further configured to determine a test expected data set corresponding to the algorithm test case by using the target mapping result and the target reference dictionary; and determine the target conversion result based on the test expected data set and the algorithm test case.

[0018] Optionally, the test module is further configured to perform the target conversion result in a distributed manner by using a target test tool to obtain a test execution result, where the target test tool is used to represent an open interface of a vehicle terminal corresponding to the vehicle voice interaction system.

[0019] Optionally, the analysis module is further configured to compare the test execution result with the test expected result to obtain a test comparison result; and analyze the test comparison result to obtain a test analysis result.

[0020] According to an embodiment of the present disclosure, an electronic device is also provided, which includes a processor, a memory for storing processor-executable instructions, and wherein the processor is configured to execute the instructions to implement the vehicle voice interaction system testing method in the embodiments of the present disclosure.

[0021] According to an embodiment of the present disclosure, a computer-readable storage medium is also provided, which includes a stored executable program, and wherein the executable program, when executed, controls a device where the storage medium is located to perform the vehicle voice interaction system testing method in the embodiments of the present disclosure.

[0022] According to an embodiment of the present disclosure, a computer program product is also provided, which includes a computer program, and the computer program, when executed by a processor, implements the vehicle voice interaction testing system method in the embodiments of the present disclosure.

[0023] In the embodiments of the present disclosure, by obtaining the target reference dictionary, and performing mapping processing by using the preset intent label and the target reference dictionary to obtain the target mapping result, then performing test processing on the vehicle voice interaction system based on the target mapping result to obtain the test execution result, and finally performing analysis processing on the test execution result to obtain the test analysis result, the purpose of optimizing the performance of the vehicle voice interaction system is achieved, thereby realizing the technical effects of improving the test efficiency and reducing the test cost, and further solving the technical problems of low test efficiency and high test cost existing in the vehicle voice interaction system testing method provided in the related art. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings, which are included to provide a further understanding of the present disclosure and constitute a part of the present disclosure, illustrate the exemplary embodiments of the present disclosure and specifically explain the present disclosure together with the description. In the drawings:

[0025] Figure 1is a flow chart of a vehicle voice interaction test system method according to an embodiment of the present disclosure;

[0026] Figure 2 is a schematic diagram of a vehicle voice interaction system test method according to an embodiment of the present disclosure;

[0027] Figure 3 is a structural block diagram of a vehicle voice interaction test system device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to enable persons skilled in the art to better understand the present disclosure scheme, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by persons skilled in the art without creative labor should be within the scope of protection of the present disclosure.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0030] The vehicle voice interaction system test method in the related art usually adopts functional testing or model testing to realize vehicle voice interaction system testing, wherein the functional testing focuses on verifying the execution accuracy of a single function, ensuring that each function can accurately respond to user voice instructions, and the model testing focuses on evaluating the performance of the algorithm, such as recall rate and accuracy, to optimize the understanding and processing of the algorithm to user voice instructions. However, the two test methods in the related art have the problems of low test efficiency and high test cost, thereby making it difficult to realize large-scale vehicle voice interaction system testing.

[0031] Specifically, model testing can evaluate the recall and accuracy of the algorithm, but this testing method often cannot go deep into the complete flow from the user issuing an instruction to the actual function execution. This limitation leads to the existence of blind spots in the voice interaction link that are not fully detected in the batch testing process, which can adversely affect the final product performance and user experience. On the other hand, although functional testing can cover the complete path of voice execution to ensure the comprehensiveness of the test, this method usually requires human intervention, which not only reduces the efficiency of the test, but also increases the cost of the test. In the modern fast-paced product development cycle, the above two testing methods with low testing efficiency and high testing cost are difficult to meet the market demand for rapid iteration and high-quality output.

[0032] According to an embodiment of the present disclosure, a method embodiment of a vehicle voice interaction system testing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown.

[0033] The method embodiment can be executed in an electronic device or similar computing device containing a memory and a processor. Taking an example of running on a computer terminal, the computer terminal can include one or more processors (the processor can include but is not limited to a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP) chip, a microcontroller unit (MCU), a field programmable gate array (FPGA), a neural-network processor unit (NPU), a tensor processing unit (TPU), an artificial intelligence (AI) type processor, etc.) and a memory for storing data. Optionally, the above computer terminal can also include a transmission device for communication function, an input and output device, and a display device. Those skilled in the art can understand that the above structural description is only illustrative, and it does not limit the structure of the above computer terminal. For example, the computer terminal can include more or less components than the above structural description, or have a different configuration from the above structural description.

[0034] The memory can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the vehicle voice interaction system testing method in the embodiments of the present disclosure. The processor executes various functions and data processing by running the computer programs stored in the memory, that is, implements the vehicle voice interaction system testing method described above. The memory can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged with respect to the processor, which can be connected to the mobile terminal through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0035] The transmission device is used to receive or send data via a network. The specific examples of the above-mentioned network can include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, RF) module used for communicating with the Internet in a wireless manner.

[0036] The display device can be, for example, a liquid crystal display (Liquid Crustal Display, LCD) and a touch display (also referred to as "touch screen" or "touch display screen") in the form of a touch screen. The liquid crystal display can enable a user to interact with a user interface of the mobile terminal. In some embodiments, the above-mentioned mobile terminal has a graphical user interface (Graphical User Interface, GUI), and a user can interact with the GUI through finger contact and / or gestures on a touch-sensitive surface. The human-computer interaction function here can optionally include the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital videos, playing digital music, and / or web browsing, etc. Executable instructions for performing the above human-computer interaction functions are configured / stored in one or more computer program products or readable storage media executable by the processor.

[0037] Figure 1 is a flowchart of a vehicle voice interaction system testing method according to an embodiment of the present disclosure, as shown in Figure 1 The method comprises the following steps:

[0038] Step S11, obtaining a target reference dictionary, wherein the target reference dictionary is used to represent the mapping relationship between the function test case and the vehicle test log, the function test case is used to verify whether at least one voice interaction function is executed according to the function expected result, and the vehicle test log is used to determine the data flow record of the voice interaction link;

[0039] Step S12, mapping processing is performed by using the preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine the mapping relationship between the function test case and the algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to the algorithm expected result;

[0040] Step S13, performing test processing on the vehicle voice interaction system based on the target mapping result to obtain a test execution result;

[0041] Step S14, performing analysis processing on the test execution result to obtain a test analysis result.

[0042] The target reference dictionary is a detailed database or data set, which is used to record and standardize the key data points and expected results in the whole process from the voice assistant parsing, understanding, execution to feedback of the user voice instruction, and is used to represent the mapping relationship between the function test case and the vehicle test log. The generation process of the target reference dictionary includes three parts of use case collection, real vehicle recording and log analysis.

[0043] The function test case is a test case used to verify whether a specific function of a software system is normally operated according to the specification or user demand, mainly focuses on the user voice instruction input, the actual reaction of the vehicle and the voice feedback output, and is a typical black box test method. The purpose of the function test case is to verify the integrity and correctness of the voice interaction link.

[0044] Exemplarily, when performing real vehicle testing, for each specific function point, only one representative voice instruction needs to be verified to check whether the entire interaction link from user voice input to vehicle terminal function execution is smooth and the final execution result is consistent with the expectation. For example, for vehicle control functions, the system can be verified whether it can correctly understand the instruction and execute the corresponding control function by issuing the instruction of "opening the window" to judge whether the vehicle executes the action of "opening the window" and whether it replies the voice feedback content of "good, I have opened it for you". At the same time, it is ensured that the text-to-speech (TTS) reply output is accurate and the instruction is clear. For navigation functions, the system can be verified whether it can correctly understand the instruction, call the navigation algorithm, plan the route and finally issue the correct navigation execution signal by issuing the instruction of "going to the nearest gas station". At the same time, it is ensured that the TTS reply output is accurate and the instruction is clear. In this way, not only potential problems in the voice interaction link can be detected, but also the effectiveness and comprehensiveness of the test can be ensured under the minimum test set, so as to reduce the consumption of test resources while ensuring the test quality and improve the overall test efficiency.

[0045] The vehicle test log, i.e. the real vehicle log, is a collection of data and information recorded by the electronic control system, software system or special test equipment of the vehicle during the test of the vehicle system or component. The vehicle test log not only records the receiving, understanding and execution process of the voice instruction, but also contains the actual response of the vehicle to the above-mentioned command and the interaction between the voice assistant and other systems of the vehicle. Through the analysis of the vehicle test log, the functional integrity, performance, user experience of the voice assistant and the integration effect with the vehicle system can be evaluated, so as to identify and solve potential problems.

[0046] The data flow record, i.e. the analysis of the real vehicle log, captures and records the dynamic behavior of the voice interaction function from the input of the user voice instruction to the execution of the vehicle terminal function by analyzing the log generated when the function test case is executed in the real vehicle environment.

[0047] The preset intent label, i.e. the intent of the semantic understanding model, is defined as a unified label for the relationship mapping of the function point. In the test process, the preset intent label is used to map the user voice instruction to its intended function point, so as to automatically execute the related test case.

[0048] Exemplarily, in the voice assistant test of the intelligent cockpit, the preset intent labels can include "navigate to…", "adjust air conditioning temperature", "play music", etc., each label corresponding to one or more specific voice instructions and expected system responses. By matching the instructions in the algorithm test cases with their preset intent labels, it can be ensured that the test covers the entire end-to-end flow of the voice assistant from receiving the instruction to executing the function, thereby verifying the accuracy and reliability of the voice assistant.

[0049] The above mapping process includes two parts of label definition and mapping rule, wherein the label refers to the smallest unit of distinguishing each function point; the label definition refers to taking the intent label of the semantic understanding model as the benchmark for function point relationship mapping, so that all test cases are classified and identified based on the label, for mapping the user voice instruction and its expected input and output of the benchmark dictionary to the user's statement under the same function point of the algorithm test set; the mapping rule refers to mapping the algorithm test case and the function test case according to the unified label, ensuring that each algorithm test case can be mapped to the function test case and its corresponding vehicle precondition, test execution result.

[0050] The above algorithm test case generally refers to a set of test inputs and expected outputs designed to verify the correctness and performance of the algorithm, mainly focusing on the performance and accuracy of core algorithms such as speech recognition, semantic understanding, and dialogue management, and is a kind of white box test. The algorithm test case includes but is not limited to algorithm input, expected algorithm output, and evaluation index. Unlike the function test case, the algorithm test case focuses more on verifying the performance of the algorithm under ideal or controlled conditions, without involving the interaction of actual vehicle hardware or user interface.

[0051] Exemplarily, the Application Programming Interface (API) is used to test whether the intent understood by the Natural Language Understanding (NLU) model is accurate. For example, "open the window", it is judged whether the NLU result is mapped to the algorithm model output action = WindowsOpen, the algorithm model output slot = { "set_type": "open again", "device": "window", "value": "a little"}. By comparing the intent recognition result of the model output with the preset correct answer, it is tested whether the understood intent is accurate, thereby effectively identifying and locating the possible problems of the model in understanding the user's instruction, and further ensuring the functional accuracy of the voice assistant in the intelligent cockpit environment and the user experience.

[0052] The vehicle voice interaction system described above is an intelligent cockpit voice assistant that performs batch testing. The intelligent cockpit voice assistant is a technology integrated in new energy vehicles that provides a convenient interactive experience by understanding and executing voice commands from users. The intelligent cockpit voice assistant uses advanced voice recognition and semantic understanding technology in combination with vehicle control logic to achieve voice control of vehicle functions such as navigation, music playback, and vehicle setting adjustments, thereby enhancing user experience and making the driving process more intelligent and personalized.

[0053] Based on the steps S11 to S14, the target reference dictionary is obtained; the target mapping result is obtained by mapping the preset intent label and the target reference dictionary; the test execution result is obtained by performing test processing on the vehicle voice interaction system based on the target mapping result; and the test analysis result is obtained by analyzing the test execution result, thereby achieving the purpose of optimizing the performance of the vehicle voice interaction system, realizing the technical effects of improving the test efficiency and reducing the test cost, and further solving the technical problems of low test efficiency and high test cost in the vehicle voice interaction system test method provided in the related art.

[0054] The vehicle voice interaction system test method in the embodiments of the present disclosure will be further introduced below.

[0055] Optionally, in step S11, obtaining the target reference dictionary comprises:

[0056] In step S111, a function test case is obtained, wherein the function test case includes a user voice command, a vehicle precondition, an expected execution result, and function description information.

[0057] In step S112, end-to-end function testing is performed based on the function test case to obtain a vehicle test log.

[0058] In step S113, the vehicle test log is parsed to obtain a log parsing result, wherein the log parsing result is used to extract a data flow record of a voice interaction link from the vehicle test log.

[0059] In step S114, the target reference dictionary is constructed according to the function test case and the log parsing result.

[0060] The above function test case includes a user voice instruction, a vehicle precondition, an expected execution result, and function description information. The user voice instruction refers to a voice instruction issued by a user to a vehicle intelligent system through natural language or a specific command. The voice instruction is the starting point of user interaction with the vehicle intelligent system and is used to convey the user's needs. The vehicle precondition, that is, the car machine perception point input, refers to a series of conditions that need to be met by the vehicle intelligent system before executing the user voice instruction, to ensure that the instruction is executed in a proper environment to avoid potential risks or errors. The expected execution result refers to the specific action or state reached by the vehicle intelligent system after the user issues the voice instruction. The function description information is a detailed description of a specific function of the vehicle intelligent system, which can help the tester understand the system design intent and ensure that the test case can fully cover various possible use cases, thereby verifying the integrity and reliability of the function.

[0061] Table 1 is a function test case according to an embodiment of the present disclosure. As shown in Table 1, the function test case includes a client test case and a unified tag. The client test case includes a test case title, a test premise, a test step, and an expected result.

[0062] For example, the test premise of the test case title "Turn on extreme cooling when the outside temperature is greater than 30 degrees" is "1. The network is connected, 2. In any interface, 3. Panoramic voice is turned on, 4. The outside temperature is greater than 30 degrees". The test step is to issue a "maximum air conditioning" voice command. The expected result is to receive a "extreme cooling is turned on" voice feedback and implement the "turn on extreme cooling" function. The unified tag of the above client test case is defined as "ac_hvac_max", indicating that this user instruction aims to require the vehicle air conditioning system to reach the maximum cooling capacity, that is, to turn on the extreme cooling mode.

[0063] For example, the user voice instruction includes but is not limited to specific commands such as "turn on air conditioning" or "navigate to the nearest gas station". The vehicle precondition includes but is not limited to the current running state of the vehicle, such as whether the engine is started, whether the vehicle is in a stationary or driving state, external environmental information such as weather conditions and road conditions, and system personalization settings such as user preference settings and historical operation records. For example, the user voice instruction "lie down on the seat" will first judge the current vehicle state before issuing the execution action. If the vehicle supports the "lie down on the seat" function, it will output the action of lying down on the seat and the corresponding reply. If it does not support, it will not execute the corresponding action and reply "does not support the function".

[0064] Table 1 function test case

[0065]

[0066] The above-mentioned end-to-end function test, i.e., end-to-end function test in a real vehicle, records the preconditions and execution process of the use case through the log of the real vehicle. The end-to-end function test method is different from the traditional function test or model test, and mainly focuses on the test of the entire chain from the user issuing a voice instruction to the vehicle system finally executing the instruction, to ensure the coherence and correctness of the links such as voice recognition, semantic understanding, function calling and execution feedback.

[0067] Exemplarily, before the test is performed, first, the vehicle preconditions are set, such as vehicle position, vehicle speed, vehicle temperature, entertainment system state, etc., and the vehicle preconditions will be used as the background environment for the execution of the voice instruction; second, the voice instruction is issued to the voice assistant in the vehicle, such as "navigate to the airport", "adjust the air conditioner temperature to 22 degrees", etc.; finally, the vehicle system records the user voice instruction, vehicle preconditions, expected execution result and function description information, to obtain the vehicle test log.

[0068] Table 2(a) is a vehicle log analysis result according to one embodiment of the present disclosure. As shown in Table 2(a), the vehicle log analysis result includes voice command, voice command sound area, car machine end perception point input and algorithm model output action. When the target reference dictionary is obtained, the uniform label corresponding to the vehicle log analysis result needs to be defined.

[0069] Table 2(a) vehicle log analysis result

[0070]

[0071] Table 2(b) is a vehicle log analysis result according to one embodiment of the present disclosure. As shown in Table 2(b), the vehicle log analysis result includes algorithm model output slot, car machine end execution command output and car machine end voice reply output. When the target reference dictionary is obtained, the uniform label corresponding to the vehicle log analysis result needs to be defined.

[0072] Table 2(b) vehicle log analysis result

[0073]

[0074] Exemplarily, taking the use case title "Outside temperature greater than 30 degrees - turn on extreme cooling" as an example, a "maximum air conditioner" voice command is issued, the voice assistant executes the relevant test steps according to the received voice command, and the current real vehicle log is parsed to obtain the following results: the voice area where the voice command is located is a low-frequency voice area, the car machine end perception point input is " {"ac.outside.temp": {"value": 31}}", the algorithm model output action is "AcSet", the algorithm model output slot is " {"set_type": "adjust to", "device": "air conditioner", "value": "maximum"}", the car machine end execution command output is " ['ac.fast.warming.on'] ", and the car machine end voice reply output is "extreme cooling is turned on".

[0075] Specifically, to implement batch end-to-end voice interaction testing, a series of functional test cases are first obtained from the voice assistant function test of the intelligent cockpit, which includes user-issued voice instructions such as "turn on the air conditioner and set the temperature to 22 degrees", vehicle preconditions such as the vehicle being in an idle state and the air conditioning system being available, and expected execution results and description information of the function, such as the air conditioner adjusting to 22 degrees according to the instruction. Subsequently, based on the above functional test cases, end-to-end functional testing is performed in the real vehicle environment of the new energy vehicle. During testing, the vehicle's voice interaction system will respond to user instructions, execute corresponding functions, and record the entire interaction process, including voice recognition, semantic analysis, function execution feedback, etc., forming detailed vehicle test logs. After obtaining the real vehicle test logs, they are parsed and processed to extract key information, forming log analysis results. The log analysis results include the input of the vehicle perception point, i.e., the vehicle preconditions, the input of the user voice instruction, the output of the semantic understanding model, the output of the car machine execution signal, and the TTS reply output. Finally, according to the functional test cases and the log analysis results, a target benchmark dictionary is constructed. The benchmark dictionary will contain detailed information of the above test cases and data stream records obtained in the real vehicle test, serving as a basis for subsequent batch end-to-end testing. For example, for the instruction "turn on the air conditioner and set the temperature to 22 degrees", the benchmark dictionary will record how the semantic understanding model correctly parses the instruction under certain vehicle preconditions, how the car machine system responds and executes, and how the voice assistant feeds back to the user. By constructing the benchmark dictionary during batch testing, it can be ensured that the algorithm test cases can be accurately converted into end-to-end test cases, covering the entire process from user instructions to function execution, thereby improving testing efficiency.

[0076] Based on the steps S111 to S114, the vehicle test log is obtained by acquiring the function test case, performing end-to-end function test based on the function test case, and then performing analysis processing on the vehicle test log to obtain the log analysis result. Finally, the target reference dictionary is constructed according to the function test case and the log analysis result, which can systematically record and standardize the corresponding relationship between the voice instruction and the function execution, thereby improving the test efficiency.

[0077] Optionally, in step S12, mapping processing is performed using the preset intent label and the target reference dictionary to obtain a target mapping result, including:

[0078] In step S121, a first mapping relationship is determined based on the preset intent label and the algorithm test case, wherein the first mapping relationship is used to represent the mapping relationship between the first function point corresponding to the algorithm test case, the first voice instruction, and the preset intent label.

[0079] In step S122, a second mapping relationship corresponding to the function test case is determined based on the target reference dictionary, wherein the second mapping relationship is used to represent the mapping relationship between the second function point corresponding to the function test case, the second voice instruction, and the preset intent label.

[0080] In step S123, the target mapping result is determined using the first mapping relationship and the second mapping relationship.

[0081] The first mapping relationship refers to the mapping relationship between the algorithm test set and the preset intent label, and is used to represent the mapping relationship between the first function point corresponding to the algorithm test case, the first voice instruction, and the preset intent label. The main target is to map the intent, voice instruction, and expected output in the algorithm test case to a unified label system, ensuring that the algorithm-level test can be accurately and correctly connected with the actual function execution test in batch testing.

[0082] The second mapping relationship refers to the mapping relationship between the client case and the preset intent label, and is used to represent the mapping relationship between the second function point corresponding to the algorithm test case, the second voice instruction, and the preset intent label. The user voice instruction, vehicle precondition, and expected result obtained through real vehicle testing establish a mapping between the function test case and the preset intent label in the algorithm test case, ensuring that the function test case can be accurately converted into the required form of the algorithm test case for end-to-end batch testing.

[0083] Exemplarily, the algorithm test set includes a first function point and a first voice instruction. Assuming that the first function point is "air conditioner maximum", and the first voice instruction is "air conditioner open to maximum", "air conditioner open to maximum", "air conditioner open to maximum", "air conditioner open to maximum", and "air conditioner maximum gear", etc. related instructions, set its uniform label as "ac_hvac_max", thereby constructing "air conditioner maximum-air conditioner open to maximum-ac_hvac_max" and other first mapping relationship. The client use case includes a second function point and a second voice instruction. Assuming that the second function point is "air conditioner maximum", and the second voice instruction is "air conditioner open to maximum", set its uniform label as "ac_hvac_max", thereby constructing "air conditioner maximum-air conditioner open to maximum-ac_hvac_max" second mapping relationship.

[0084] Table 3 is a uniform label mapping result according to an embodiment of the present disclosure. As shown in Table 3, through the uniform label, the mapping relationship between the algorithm test case and the function test case can be realized.

[0085] Table 3 uniform label mapping result

[0086]

[0087] Exemplarily, taking the client use case function point of "air conditioner maximum" as an example, the corresponding voice command can be "air conditioner open to maximum", and its uniform label is defined as "ac_hvac_max". For the algorithm test set with the function point of "air conditioner maximum", the corresponding voice command can be "air conditioner open to maximum", "air conditioner open to maximum", "air conditioner open to maximum", etc., and its uniform label is defined as "ac_hvac_max", thereby according to the same uniform relationship, the algorithm test case and the function test case can be mapped.

[0088] Based on the above steps S121 to S123, by determining the first mapping relationship based on the preset intent label and the algorithm test case, determining the second mapping relationship corresponding to the function test case based on the target reference dictionary, and determining the target mapping result by using the first mapping relationship and the second mapping relationship, the algorithm test case and the function test case can be effectively associated, ensuring that each algorithm test case can be accurately mapped to the corresponding function test case and its precondition and execution result, thereby ensuring the comprehensiveness of the end-to-end test, reducing the demand for real vehicle test, reducing the labor cost, and improving the test efficiency.

[0089] Optionally, in step S13, based on the target mapping result, the vehicle voice interaction system is executed for test processing, and the test execution result is obtained, including:

[0090] In step S131, the algorithm test case is converted based on the target mapping result, and the target conversion result is obtained.

[0091] Step S132, using the target conversion result to perform test processing on the vehicle voice interaction system, obtaining a test execution result.

[0092] The above conversion processing refers to converting algorithm test cases into end-to-end test cases, the core of which is to extend the test from the isolated algorithm level to the test of the entire system flow, ensuring that each link from user input to system response is accurately tested. Through label mapping, the algorithm case is added with end-to-end expectations, and there are various voice instructions under the same function, i.e., user statements, so batch testing and verification can be performed.

[0093] The above target conversion result, i.e., the end-to-end test case, through the construction of the benchmark dictionary and the mapping of the unified label, can integrate the originally scattered algorithm test cases and function test cases to form a complete set of test case collection for the whole process of voice assistant from user voice instruction reception to function execution.

[0094] Table 4(a) is a batch test case according to an embodiment of the present disclosure. As shown in Table 4(a), the batch test case includes an algorithm test set, a unified label, and a mapped end-to-end expectation, wherein the mapped end-to-end expectation includes a case title, a test premise, an expected result, and an audio area.

[0095] Table 4(b) is a batch test case according to an embodiment of the present disclosure. As shown in Table 4(b), the batch test case includes an algorithm test set, a unified label, and a mapped end-to-end expectation, wherein the mapped end-to-end expectation includes a car machine end perception point input, an algorithm model expected action, an algorithm model expected slot, a car machine end expected execution command, and a car machine end expected voice reply.

[0096] Table 4(a) batch test case

[0097]

[0098] Table 4(b) batch test case

[0099]

[0100] Exemplarily, taking the voice command "air conditioner open to maximum" as an example, its uniform tag is "ac_hvac_max", and through uniform tag mapping, its mapped end-to-end expectation is obtained, and the end-to-end expectation result includes: the use case title is "outside temperature greater than 30 degrees - start extreme cooling", the test premise is "1. The network is connected; 2. In any interface; 3. Full scene voice is on; 4. The outside temperature is greater than 30 degrees", the expected result is "1. Voice feedback: extreme cooling is turned on; 2. Function: start extreme cooling", the voice command is in the "low frequency sound area", the car machine end sensing point input is "{"ac.outside.temp": {"value": 31}}", the algorithm model output expected action is "AcSet", the algorithm model output expected slot is "{"set_type": "adjust to", "device": "air conditioner", "value": "maximum"}", the car machine end expected execution command is "['ac.fast.warming.on']", and the car machine end expected voice reply is "extreme cooling is turned on". According to the batch test case, the recorded use case can be regressed through the end-to-end test tool, and does not need to be verified on the real vehicle again, so as to reduce the real vehicle test manpower cost. In addition, since the pure model test only verifies the "brain" of the voice assistant, and the test conclusion cannot represent the real user experience, the action execution needs the cooperation of each link of the voice link, and the batch test is no longer limited to the NLU model test, and the conclusion of the end-to-end test can verify each link, so the test conclusion can better represent the real vehicle user experience and has higher credibility.

[0101] Specifically, the intention of the algorithm test case is corresponded to the function point in the benchmark dictionary, the mapping is realized by defining a uniform tag, each algorithm test case not only includes the expected input and output at the algorithm level, but also is associated with the preconditions and expected execution results of the real vehicle test, and becomes a complete end-to-end test case. Finally, an automatic test tool is used to perform the test. It is ensured that the test tool can accept the user voice instruction and the car machine sensing point as the input, simulate the response process of the voice assistant in the actual use environment, collect and analyze the test results, and compare with the expected output to evaluate the performance and reliability of the voice assistant.

[0102] Based on the steps S131 to S132, the algorithm test case is converted based on the target mapping result to obtain a target conversion result, and the vehicle voice interaction system is tested based on the target conversion result to obtain a test execution result, which can quickly cover a large number of test scenes, reduce the dependence on real vehicle resources, and reduce the test cost.

[0103] Optionally, in step S131, the algorithm test case is converted based on the target mapping result to obtain a target conversion result, including: determining the test expected data set corresponding to the algorithm test case by using the target mapping result and the target reference dictionary; and determining the target conversion result based on the test expected data set and the algorithm test case.

[0104] The test expected data set refers to a data set containing end-to-end test expected results after conversion and mapping. The test expected data set includes but is not limited to user voice instructions, vehicle perception point inputs, semantic understanding model outputs, vehicle machine execution signals, TTS replies, etc.

[0105] Exemplarily, in combination with the target mapping result and the target reference dictionary, for each algorithm test case, the expected input and the expected output thereof in the end-to-end test environment are determined to constitute the test expected data set, and the algorithm test case is converted into a form suitable for the end-to-end test environment according to the test expected data set, so as to ensure that the algorithm test case can be correctly interpreted and executed, thereby determining the target conversion result based on the test expected data set and the algorithm test case.

[0106] Based on the above optional embodiment, by determining the test expected data set corresponding to the algorithm test case by using the target mapping result and the target reference dictionary, and determining the target conversion result based on the test expected data set and the algorithm test case, it can be ensured that the algorithm test case fully covers the function link of the voice assistant, thereby achieving the purpose of batch testing.

[0107] Optionally, in step S132, the target conversion result is used to perform test processing on the vehicle voice interaction system to obtain a test execution result, including: performing distributed execution on the target conversion result by using a target test tool to obtain the test execution result, wherein the target test tool is used to represent an open interface of a vehicle machine corresponding to the vehicle voice interaction system.

[0108] The target test tool is an automatic test tool. The automatic test tool includes but is not limited to a vehicle machine open interface (Vehicle-to-Everything Application Programming Interface, V2X API), a speech recognition engine, an NLP module, a speech synthesis technology, a virtual vehicle machine environment simulator, and a test result analysis and report generation system. The automatic test tools work cooperatively to automatically execute the entire test process from user voice instruction input to function execution result output, while collecting and analyzing test data to evaluate the performance and stability of the voice assistant in the intelligent cockpit.

[0109] The distributed execution, i.e., the distributed testing architecture, is batch execution. Batch execution is performed on a test set by using an automated testing tool, ensuring that the testing tool supports user voice instructions and car machine perception point inputs as input parameters, and collecting execution results.

[0110] Exemplarily, first, communication is performed using an API interface corresponding to the target testing tool and the vehicle voice interaction system; second, based on a benchmark dictionary and a unified label mapping mechanism, batch end-to-end test cases are converted into a format that can be understood and executed by the target testing tool; and finally, the converted test cases are distributedly executed using a V2X API, and data in the entire process, including the output of the semantic understanding model, the signals executed by the car machine, and the TTS reply, are collected.

[0111] Based on the above optional embodiment, the target conversion result is distributedly executed by using the target testing tool to obtain a test execution result, which can simultaneously test based on multiple nodes, thereby improving the test efficiency and shortening the test period.

[0112] Optionally, in step S14, the test execution result is analyzed and processed to obtain a test analysis result.

[0113] In step S141, the test execution result is compared with the test expected result to obtain a test comparison result.

[0114] In step S142, the test comparison result is analyzed and processed to obtain the test analysis result.

[0115] Specifically, the test comparison result is obtained by comparing the test execution result with the test expected result of each link of the voice link. Based on the test comparison result, it is identified that the execution result of which test case does not conform to the expectation, and it is analyzed whether the deviation is from semantic understanding error, abnormal function execution signal, inaccurate TTS reply, etc. to determine the root cause of the problem. For the identified problem, it is further analyzed whether it is due to incomplete test case design, limitation of algorithm model, or execution error, and the influence of the test result on the performance of the voice assistant is evaluated, including recall rate, accuracy rate, or user experience, etc. key indicators.

[0116] Based on the above steps S141 to S142, the test execution result is compared with the test expected result to obtain the test comparison result, and the test comparison result is analyzed and processed to obtain the test analysis result, which can efficiently identify the problems encountered in the test process, and can continuously optimize the performance of the voice assistant based on the identified problems, thereby improving the experience of users in the intelligent cockpit.

[0117] Optionally, the vehicle voice interaction system testing method further comprises: uploading the target reference dictionary, the function test case and the algorithm test case to a cloud testing platform, so that the cloud testing platform performs testing processing on the vehicle voice interaction system to obtain a test execution result.

[0118] The cloud testing platform refers to an online service platform for software or hardware testing using cloud computing technology. The cloud testing platform provides virtual environments, testing tools, automated script execution, test data management, result analysis and report generation, and other functions required for testing. Users can access and use the services through the Internet without the need to build a local testing environment.

[0119] Illustratively, first, the cloud testing platform reads the test case from the reference dictionary stored in the cloud; second, the cloud testing platform uses its centralized management of automated testing tools to distribute the test case to each test node according to the predetermined execution strategy. After receiving the test case, the test node starts to perform the test steps of voice recognition, semantic understanding, and function calling. The voice recognition module is responsible for converting user voice commands into text, the semantic understanding module parses the text and determines the intent, and the function calling module executes the corresponding vehicle function according to the parsed intent. After the test is completed, each node collects the execution results, including the accuracy of voice recognition, the correctness of semantic understanding, the reliability of function execution, and the quality of TTS reply, and feeds back to the cloud testing platform.

[0120] Based on the above optional embodiment, uploading the target reference dictionary, the function test case and the algorithm test case to the cloud testing platform, and performing testing processing on the vehicle voice interaction system based on the cloud testing platform to obtain a test execution result, can reduce the dependence on real vehicle testing, reduce labor costs, and ensure the performance and user experience of the voice assistant in the intelligent cockpit.

[0121] Figure 2 is a schematic diagram of a vehicle voice interaction system testing method according to an embodiment of the present disclosure, as Figure 2As shown, in the test process of the vehicle voice interaction system, first, a benchmark dictionary is constructed through use case collection, real vehicle recording and log analysis; second, based on a unified tag mapping relationship, function point-tag definition is performed to obtain a function point-tag relationship dictionary; third, test case conversion is performed to confirm the function point tag of the algorithm test case, and according to the benchmark dictionary, the vehicle preconditions, test execution results and expected additions under the unified tag are queried to be mapped to the end-to-end batch test case; finally, based on the end-to-end batch test case, batch end-to-end test execution is performed, batch test execution is performed by configuring the end-to-end test tool, a test result file is obtained, and result analysis is performed according to the test result file to identify problems in function execution, thereby optimizing and adjusting. By constructing the benchmark dictionary and the unified tag mapping, comprehensive end-to-end test coverage is achieved, and at the same time, through automatic batch testing, the test efficiency is improved and the test cost of real vehicle testing is reduced.

[0122] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and the necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the disclosure can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes a plurality of instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of various embodiments of the disclosure.

[0123] In the embodiments of the disclosure, a vehicle voice interaction system testing device is also provided, which is used to implement the above embodiments and preferred embodiments, and will not be described again. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware or a combination of software and hardware is also possible and is contemplated.

[0124] Figure 3 is a structural block diagram of a vehicle voice interaction system testing device according to an embodiment of the disclosure, as Figure 3 shown, the device comprises:

[0125] The acquisition module 301 is configured to acquire a target benchmark dictionary, wherein the target benchmark dictionary is used to represent the mapping relationship between the function test case and the vehicle test log, the function test case is used to verify whether at least one voice interaction function is executed according to the function expected result, and the vehicle test log is used to determine the data flow record of the voice interaction link;

[0126] The mapping module 302 is configured to perform mapping processing by using the preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine a mapping relationship between the function test case and the algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to an algorithm expected result.

[0127] The test module 303 is configured to perform test processing on the vehicle voice interaction system based on the target mapping result to obtain a test execution result.

[0128] The analysis module 304 is configured to perform analysis processing on the test execution result to obtain a test analysis result.

[0129] Optionally, the acquisition module 301 is further configured to acquire the function test case, wherein the function test case includes a user voice instruction, a vehicle precondition, an expected execution result and function description information; perform end-to-end function testing based on the function test case to obtain a vehicle test log; and perform analysis processing on the vehicle test log to obtain a log analysis result, wherein the log analysis result is used to extract a data flow record of a voice interaction link from the vehicle test log; and construct the target reference dictionary according to the function test case and the log analysis result.

[0130] Optionally, the mapping module 302 is further configured to determine a first mapping relationship based on the preset intent label and the algorithm test case, wherein the first mapping relationship is used to represent a mapping relationship between a first function point corresponding to the algorithm test case, a first voice instruction and the preset intent label; determine a second mapping relationship corresponding to the function test case based on the target reference dictionary, wherein the second mapping relationship is used to represent a mapping relationship between a second function point corresponding to the function test case, a second voice instruction and the preset intent label; and determine the target mapping result by using the first mapping relationship and the second mapping relationship.

[0131] Optionally, the test module 303 is further configured to perform conversion processing on the algorithm test case based on the target mapping result to obtain a target conversion result; and perform test processing on the vehicle voice interaction system by using the target conversion result to obtain the test execution result.

[0132] Optionally, the test module 303 is further configured to determine a test expected data set corresponding to the algorithm test case by using the target mapping result and the target reference dictionary; and determine the target conversion result based on the test expected data set and the algorithm test case.

[0133] Optionally, the test module is further configured to perform distributed execution on the target conversion result by using a target test tool to obtain the test execution result, wherein the target test tool is used to represent an open interface of a vehicle machine end corresponding to the vehicle voice interaction system.

[0134] Optionally, the analysis module 304 is further configured to: compare the test execution result with the test expected result to obtain a test comparison result; and analyze the test comparison result to obtain a test analysis result.

[0135] It should be noted that the above modules can be implemented by software or hardware, and the hardware can be implemented in the following manner, but is not limited thereto: the above modules are located in the same processor; or the above modules are located in different processors in any combination.

[0136] According to an embodiment of the present disclosure, an electronic device is also provided, which includes: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the vehicle voice interaction system test method in the embodiments of the present disclosure.

[0137] Optionally, in the present embodiment, the processor can be configured to execute the following steps by means of a computer program:

[0138] Step S1, obtaining a target reference dictionary, wherein the target reference dictionary is used to represent the mapping relationship between the function test case and the vehicle test log, the function test case is used to verify whether at least one voice interaction function is executed according to the function expected result, and the vehicle test log is used to determine the data flow record of the voice interaction link;

[0139] Step S2, performing mapping processing by using the preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine the mapping relationship between the function test case and the algorithm test case, and the algorithm test case is used to verify whether the voice interaction algorithm is executed according to the algorithm expected result;

[0140] Step S3, performing test processing on the vehicle voice interaction system based on the target mapping result to obtain a test execution result;

[0141] Step S4, performing analysis processing on the test execution result to obtain a test analysis result.

[0142] According to an embodiment of the present disclosure, a computer readable storage medium is also provided, which includes a stored executable program, wherein the executable program controls the device where the storage medium is located to execute the vehicle voice interaction system test method in the embodiments of the present disclosure when the executable program is running.

[0143] Optionally, in the present embodiment, the storage medium can be configured to store a computer program for executing the following steps:

[0144] Step S1, obtaining a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a function test case and a vehicle test log, the function test case is used to verify whether at least one voice interaction function is executed according to a function expected result, and the vehicle test log is used to determine a data flow record of a voice interaction link;

[0145] Step S2, performing mapping processing by using a preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine a mapping relationship between the function test case and an algorithm test case, and the algorithm test case is used to verify whether a voice interaction algorithm is executed according to an algorithm expected result;

[0146] Step S3, performing test processing on a vehicle voice interaction system based on the target mapping result to obtain a test execution result;

[0147] Step S4, performing analysis processing on the test execution result to obtain a test analysis result.

[0148] Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various computer program storage media.

[0149] According to an embodiment of the present disclosure, a computer program product is also provided, which includes a computer program. When the computer program is executed by a processor, the vehicle voice interaction test system method in the embodiment of the present disclosure is implemented.

[0150] Optionally, in the embodiment, the computer program product can be set as a computer program for performing the following steps:

[0151] Step S1, obtaining a target reference dictionary, wherein the target reference dictionary is used to represent a mapping relationship between a function test case and a vehicle test log, the function test case is used to verify whether at least one voice interaction function is executed according to a function expected result, and the vehicle test log is used to determine a data flow record of a voice interaction link;

[0152] Step S2, performing mapping processing by using a preset intent label and the target reference dictionary to obtain a target mapping result, wherein the target mapping result is used to determine a mapping relationship between the function test case and an algorithm test case, and the algorithm test case is used to verify whether a voice interaction algorithm is executed according to an algorithm expected result;

[0153] Step S3, performing test processing on a vehicle voice interaction system based on the target mapping result to obtain a test execution result;

[0154] Step S4, analyzing the test execution result to obtain a test analysis result.

[0155] In the above-described embodiments of the present disclosure, the description of each embodiment focuses on different aspects, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0156] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technology can be implemented in other ways. Of course, the unit embodiment described above is only illustrative, and for example, the division of units can be a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, which can be electrical or other forms.

[0157] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple units. Part or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0158] In addition, each functional unit in each embodiment of the present disclosure can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0159] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present disclosure essentially or say the part that makes a contribution to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present disclosure. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0160] The above merely preferred embodiments of the present disclosure, it should be noted that for those of ordinary skill in the art, without departing from the principles of the present disclosure, can also make a number of improvements and refinements, these improvements and refinements should also be considered as the protection scope of the present disclosure.

Claims

1. A testing method for a vehicle voice interaction system, characterized in that, include: Obtain a target benchmark dictionary, wherein the target benchmark dictionary is used to represent the mapping relationship between functional test cases and vehicle test logs, the functional test cases are used to verify whether at least one voice interaction function is executed according to the expected functional result, and the vehicle test logs are used to determine the data flow records of the voice interaction link; The target mapping result is obtained by mapping the preset intent tags and the target benchmark dictionary. The target mapping result is used to determine the mapping relationship between the functional test cases and the algorithm test cases. The algorithm test cases are used to verify whether the voice interaction algorithm executes according to the expected results of the algorithm. Based on the target mapping results, test processing is performed on the vehicle voice interaction system to obtain the test execution results; The test execution results are analyzed and processed to obtain test analysis results.

2. The method according to claim 1, characterized in that, Obtaining the target benchmark dictionary includes: Obtain the functional test cases, wherein the functional test cases include: user voice commands, vehicle preconditions, expected execution results, and functional description information; End-to-end functional testing is performed based on the aforementioned functional test cases to obtain the vehicle test log; The vehicle test log is parsed to obtain a log parsing result, wherein the log parsing result is used to extract the data stream record of the voice interaction link from the vehicle test log; The target benchmark dictionary is constructed based on the functional test cases and the log parsing results.

3. The method according to claim 1, characterized in that, The target mapping result is obtained by performing mapping processing using the preset intent tags and the target baseline dictionary, including: A first mapping relationship is determined based on the preset intent tag and the algorithm test case, wherein the first mapping relationship is used to represent the mapping relationship between the first functional point and the first voice command corresponding to the algorithm test case and the preset intent tag; Based on the target benchmark dictionary, a second mapping relationship is determined corresponding to the functional test case, wherein the second mapping relationship is used to represent the mapping relationship between the second functional point, the second voice command and the preset intent label corresponding to the functional test case; The target mapping result is determined using the first mapping relationship and the second mapping relationship.

4. The method according to claim 1, characterized in that, Based on the target mapping result, test processing is performed on the vehicle voice interaction system to obtain the test execution result, which includes: Based on the target mapping result, the algorithm test cases are transformed to obtain the target transformation result; The target conversion result is used to perform test processing on the vehicle voice interaction system to obtain the test execution result.

5. The method according to claim 4, characterized in that, Based on the target mapping result, the algorithm test cases are transformed to obtain the target transformation result, including: The target mapping results and the target benchmark dictionary are used to determine the expected test dataset corresponding to the test cases of the algorithm; The target transformation result is determined based on the expected test dataset and the algorithm test cases.

6. The method according to claim 4, characterized in that, The vehicle voice interaction system is tested using the target conversion result, and the test results include: The target conversion result is distributed and executed using a target testing tool to obtain the test execution result, wherein the target testing tool is used to represent the vehicle-mounted open interface corresponding to the vehicle voice interaction system.

7. The method according to claim 1, characterized in that, The test execution results are analyzed and processed to obtain the test analysis results, including: The test execution results are compared with the expected test results to obtain the test comparison results. The test comparison results are analyzed and processed to obtain the test analysis results.

8. The method according to claim 1, characterized in that, The method further includes: The target benchmark dictionary, the functional test cases, and the algorithm test cases are uploaded to the cloud testing platform so that the cloud testing platform can perform test processing on the vehicle voice interaction system and obtain the test execution results.

9. A testing device for a vehicle voice interaction system, characterized in that, include: The acquisition module is used to acquire a target benchmark dictionary, wherein the target benchmark dictionary is used to represent the mapping relationship between functional test cases and vehicle test logs, the functional test cases are used to verify whether at least one voice interaction function is executed according to the expected functional result, and the vehicle test logs are used to determine the data flow records of the voice interaction link; The mapping module is used to perform mapping processing using preset intent tags and the target benchmark dictionary to obtain target mapping results. The target mapping results are used to determine the mapping relationship between the functional test cases and the algorithm test cases. The algorithm test cases are used to verify whether the voice interaction algorithm executes according to the expected results of the algorithm. The testing module is used to perform test processing on the vehicle voice interaction system based on the target mapping result and obtain the test execution result; The analysis module is used to analyze and process the test execution results to obtain test analysis results.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the vehicle voice interaction system testing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored executable program, wherein, when the executable program is executed, it controls the device on which the storage medium is located to perform the vehicle voice interaction system test method according to any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the vehicle voice interaction system testing method according to any one of claims 1 to 8.

Images