Vehicle-mounted voice model test method and device, terminal and storage medium

Abstract

The invention discloses a method for testing a vehicle-mounted voice model, which comprises the following steps of: S1, broadcasting a test problem of a test case in a test case set, and acquiring an answer result text of the vehicle-mounted voice model; s2, confirming a problem type corresponding to the test problem, and confirming an analysis judgment mode corresponding to the problem type; and S3, according to the analysis and judgment mode, analyzing and comparing the answer result text with a standard answer of the test question to obtain a test result of the test case. According to the test method of the vehicle-mounted voice model, the automatic test of the vehicle-mounted voice model can be realized, and the test efficiency of the vehicle-mounted voice model can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicles, and in particular, to a test method, a test device, a test terminal and a computer-readable storage medium for an in-vehicle voice model. Background Art

[0002] With the development of AI technology, large models based on voice are increasingly applied to the information entertainment systems of automobiles to enhance the user experience. In the prior art, in the test scheme of in-vehicle AI voice large models, the evaluation of the answer results of in-vehicle AI voice large models is usually realized by asking closed questions and manually reviewing the results. The tested questions have certain limitations and the test efficiency is relatively low. Summary of the Invention

[0003] The present invention aims to at least solve one of the technical problems existing in the prior art. For this reason, the present invention provides a test method for an in-vehicle voice model. The test method for the in-vehicle voice model can not only realize the automated test of the in-vehicle voice model, but also effectively improve the test efficiency of the in-vehicle voice model.

[0004] The present invention also provides a test device for an in-vehicle voice model.

[0005] The present invention also provides a test terminal.

[0006] The present invention also provides a computer-readable storage medium.

[0007] The test method for an in-vehicle voice model according to the first aspect of the present invention includes: S1, broadcasting the test question of the test case in the test case set, and obtaining the answer result text of the in-vehicle voice model; S2, confirming the question type corresponding to the test question, and confirming the analysis and determination mode corresponding to the question type; S3, analyzing and comparing the answer result text with the standard answer of the test question according to the analysis and determination mode, and obtaining the test result of the test case.

[0008] According to the test method for an in-vehicle voice model of the present invention, first, the test question of the test case in the test case set is broadcast, and the answer result text of the in-vehicle voice model is obtained. Then, the question type corresponding to the test question is confirmed, and the analysis and determination mode corresponding to the question type is confirmed. After that, according to the analysis and determination mode, the answer result text is analyzed and compared with the standard answer of the test question, and the test result of the test case is obtained. Thus, not only can the automated test of the in-vehicle voice model be realized, but also the test efficiency of the in-vehicle voice model can be effectively improved.

[0009] In some embodiments, the question types include: solution and explanation type, recommendation type, and vehicle control type.

[0010] In some embodiments, when the question type of the test question is of the answer - explanation type, the standard answer is the standard answer of the answer type, and step S3 includes: comparing the answer result text with the standard answer of the answer type to obtain the semantic similarity between the answer result text and the standard answer of the answer type; determining whether the semantic similarity meets a preset condition. If so, it is determined that the test result of the test case is a pass. If not, it is determined that the test result of the test case is a fail.

[0011] In some embodiments, when the question type of the test question is of the recommendation type, the standard answer is a set of information verification points, and the set of information verification points includes multiple information verification points. The information verification point is an attribute feature of the recommended target. Step S5 includes: generating summary information of the answer result text; extracting recommended items from the summary information; obtaining a detailed information set of the recommended items; determining whether the detailed information set is consistent with the set of information verification points. If so, it is determined that the test result of the test case is a pass. If not, it is determined that the test result of the test case is a fail.

[0012] In some embodiments, when the test question is a music - recommendation - type question, the multiple information verification points in the set of information verification points include: singer, song name, music style, album, and / or release year; when the test question is a movie - recommendation - type question, the multiple information verification points in the set of information verification points include: director, actor, release year, award - winning situation, rating, type, and / or running time; when the test question is a scenic - spot - recommendation - type question, the multiple information verification points in the set of information verification points include: city, charge, and / or scenic - spot category.

[0013] In some embodiments, when the question type of the test question is of the vehicle - control type, the standard answer is a set of execution - result checkpoints, and the set of execution - result checkpoints includes multiple execution - result status parameters. The execution - result status parameter is the target status parameter of the vehicle after executing the test question. Step S5 includes: obtaining a set of status parameters of the control node corresponding to the test question through the CAN bus; determining whether the set of status parameters is consistent with the set of execution - result checkpoints. If so, it is determined that the test result of the test case is a pass. If not, it is determined that the test result of the test case is a fail.

[0014] In some embodiments, before step S2, the test method further includes: retrieving whether the answer result text contains keywords corresponding to the test question. If so, step S2 is executed. If not, it is determined that the test result of the test case is a fail.

[0015] In some embodiments, the test case set includes multiple test cases. After step S3, the test method further includes: S4, repeating steps S1 - S3 until all the test cases are traversed; S5, analyzing all the test results to obtain the test conclusion of the in-vehicle voice model.

[0016] In some embodiments, step S5 includes: calculating the test pass rate of multiple test cases in the test case set; when the test pass rate is greater than or equal to the preset pass rate, the in-vehicle voice model passes the test.

[0017] The test device for the in-vehicle voice model according to the second aspect of the present invention includes: a broadcast module for broadcasting the test questions of the test cases in the test case set; an acquisition module for acquiring the answer result text of the in-vehicle voice model; a confirmation module for confirming the question type corresponding to the test question and the analysis and determination mode corresponding to the question type; an analysis and comparison module for analyzing and comparing the answer result text with the standard answer of the test question; a determination module for judging the test result of the test case and for judging the test conclusion of the in-vehicle voice model.

[0018] The test device for the in-vehicle voice model according to the second aspect of the present invention, by setting a broadcast module, an acquisition module, a confirmation module, an analysis and comparison module, and a determination module in the test device, where the broadcast module is used to broadcast the test questions of the test cases in the test case set, the acquisition module is used to acquire the answer result text of the in-vehicle voice model, the confirmation module is used to confirm the question type corresponding to the test question and the analysis and determination mode corresponding to the question type, the analysis and comparison module is used to analyze and compare the answer result text with the standard answer of the test question, and the determination module is used to judge the test result of the test case and to judge the test conclusion of the in-vehicle voice model, can not only realize the automated test of the in-vehicle voice model, but also effectively improve the test efficiency of the in-vehicle voice model.

[0019] The test terminal according to the third aspect of the present invention, the test terminal includes a memory, a processor, and a test program for the in-vehicle voice model stored on the memory and executable on the processor. When the test program for the in-vehicle voice model is executed by the processor, it implements the steps of the test method for the in-vehicle voice model according to the first aspect of the present invention.

[0020] The test terminal according to the third aspect of the present invention, by setting a memory, a processor, and a test program for the in-vehicle voice model stored on the memory and executable on the processor, and when the test program for the in-vehicle voice model is executed by the processor, it implements the steps of the test method for the in-vehicle voice model, can not only realize the automated test of the in-vehicle voice model, but also effectively improve the test efficiency of the in-vehicle voice model.

[0021] A computer-readable storage medium according to the fourth aspect of the present invention stores a test program for a vehicle-mounted voice model on the computer-readable storage medium. When the test program for the vehicle-mounted voice model is executed by a processor, the steps of the test method for the vehicle-mounted voice model according to the first aspect of the present invention are implemented.

[0022] According to the computer-readable storage medium of the fourth aspect of the present invention, by storing a test program for a vehicle-mounted voice model on the computer-readable storage medium, when the test program for the vehicle-mounted voice model is executed by a processor, the steps of the test method for the vehicle-mounted voice model according to the first aspect of the present invention are implemented, which can not only realize the automated test of the vehicle-mounted voice model, but also effectively improve the test efficiency of the vehicle-mounted voice model.

[0023] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. Description of the Drawings

[0024] Figure 1 is a flowchart of a test method for a vehicle-mounted voice model according to an embodiment of the present invention;

[0025] Figure 2 is a flowchart of compiling a test case set according to a specific example of the present invention;

[0026] Figure 3 is a flowchart of a test method for a vehicle-mounted voice model according to a specific example of the present invention;

[0027] Figure 4 is a schematic diagram of a test device for a vehicle-mounted voice model according to an embodiment of the present invention;

[0028] Figure 5 is a schematic diagram of a test terminal according to an embodiment of the present invention.

[0029] Reference Signs:

[0030] 100, test device;

[0031] 10, broadcast module; 20, acquisition module; 30, confirmation module; 40, analysis and comparison module; 50, determination module;

[0032] 600, test terminal;

[0033] 61, non-volatile storage medium; 611, operating system; 612, test program; 62, internal memory; 70, processor;

[0034] 700, system bus. Detailed Embodiment

[0035] Embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present invention, and should not be construed as limiting the present invention.

[0036] Reference will be made below to Figure 1 describe a test method for an in-vehicle voice model according to an embodiment of the first aspect of the present invention.

[0037] As Figure 1 shown, the test method for an in-vehicle voice model according to an embodiment of the first aspect of the present invention includes:

[0038] S1. Announce the test questions of the test cases in the test case set, and obtain the answer result text of the in-vehicle voice model.

[0039] In some specific examples, first select a test case from the pre-prepared test case set. Further, each test case should contain a clear test question or command. Then, announce the test question or command of the test case by voice. After that, the voice recognition system of the in-vehicle voice model can effectively recognize the announced voice signal, convert the voice signal into text and input it into the in-vehicle voice model for processing. The in-vehicle voice model can generate and output the answer result text according to the test question or command text. Thus, the answer result text of the in-vehicle voice model corresponding to the test question or command of the test case in the test case set announced by voice can be obtained.

[0040] S2. Confirm the question type corresponding to the test question, and confirm the analysis and determination mode corresponding to the question type.

[0041] It should be noted that each test question corresponds to its own question type, and each question type corresponds to a corresponding analysis and determination mode. That is to say, when using the test method of the in-vehicle voice model of the present application to test the in-vehicle voice model, the analysis and determination mode corresponding to the question type of the test question can be automatically selected according to the question type of the test question, so as to analyze and determine the test question specifically. Thus, the reliability and evaluation efficiency of the analysis and determination result of the test question can be effectively improved.

[0042] In some specific examples, the test questions include closed - ended questions and open - ended questions. Both closed - ended questions and open - ended questions correspond to corresponding question types. That is to say, both closed - ended questions and open - ended questions correspond to corresponding analysis and determination modes. Thus, the scope of test questions can be effectively expanded, and the applicability of the test method for in - vehicle voice models can be effectively improved. In addition, the analysis and determination of open - ended questions can be effectively realized, thereby effectively increasing the intelligence level of the test method for in - vehicle voice models.

[0043] S3. According to the analysis and determination mode, analyze and compare the answer result text with the standard answer of the test question to obtain the test result of the test case.

[0044] It should be noted that each test question is preset with a corresponding standard answer. The standard answer is set on the premise of correctly understanding the test question or command. Further, the standard answer can include multiple correct options to adapt to different but correct response methods, thereby further ensuring the accuracy of the test result.

[0045] In some specific examples, after using the test method of the in - vehicle voice model of the present application to select the analysis and determination mode corresponding to the test question, according to the selected analysis and determination mode, analyze and compare the answer result text with the standard answer of the test question, and comprehensively determine whether the test result meets the expectation.

[0046] According to the test method of the in - vehicle voice model of the embodiment of the present invention, first broadcast the test question of the test case in the test case set, obtain the answer result text of the in - vehicle voice model, then confirm the question type corresponding to the test question, confirm the analysis and determination mode corresponding to the question type. After that, according to the analysis and determination mode, analyze and compare the answer result text with the standard answer of the test question to obtain the test result of the test case. Thus, not only can the automated test of the in - vehicle voice model be realized, but also the test efficiency of the in - vehicle voice model can be effectively improved.

[0047] In an embodiment of the present invention, the test case set is manually written according to the function of the in - vehicle voice model. Each test case includes a test question, keywords for answering the test question, the question type corresponding to the test question, and the standard answer of the test question.

[0048] In some specific examples, writing the test case set according to the function of the in - vehicle voice model can effectively improve the pertinence of the test case set, thereby effectively improving the accuracy and reliability of the evaluation of the response ability of the in - vehicle voice model in a real - world scenario. Further, since the test case set is manually written, it can effectively avoid the deviation between the test case set and the test requirements caused by non - manually written test case sets, so as to ensure that the test case set is more in line with the test requirements, and further effectively improve the accuracy of the test conclusion.

[0049] In addition, as the functions of in-vehicle voice models are extended and improved, the test case set also needs to be updated accordingly, adding new test cases and removing test cases that are no longer relevant. In some specific examples, synonyms can be included in the keywords for answering test questions, thereby further improving the accuracy of test conclusions.

[0050] In this embodiment, the test case set is manually written according to the functions of the in-vehicle voice model. Each test case includes a test question, keywords for answering the test question, the question type corresponding to the test question, and the standard answer to the test question, which can effectively improve the accuracy and reliability of the test conclusions for the in-vehicle voice model.

[0051] In one embodiment of the present invention, the question types include: solution and explanation types, recommendation types, and vehicle control types.

[0052] It should be noted that solution and explanation type questions usually require the in-vehicle voice model to provide detailed information or detailed explanations related to the question. For example, a solution and explanation type question can be "What is the situation of today's weather forecast?"; Another example is that a solution and explanation type question can be "What are the details of today's news events?". When users seek advice or preference selections, recommendation type questions are usually used. For example, a recommendation type question can be "Are there any good places to eat nearby?"; Another example is that a recommendation type question can be "Play some relaxing music". Vehicle control type questions usually involve operation instructions for vehicle internal devices. For example, a vehicle control type question can be "Adjust the seat position forward"; Another example is that a vehicle control type question can be "Open the front row windows".

[0053] In this embodiment, by setting the question types to solution and explanation types, recommendation types, and vehicle control types, the range of types of test questions for test cases can be effectively increased, thereby effectively improving the applicability and comprehensiveness of the test method for the in-vehicle voice model.

[0054] In one embodiment of the present invention, when the question type of the test question is of the solution and explanation type, the standard answer is the solution type standard answer. Step S3 includes: comparing the answer result text with the solution type standard answer to obtain the semantic similarity between the answer result text and the solution type standard answer; determining whether the semantic similarity meets the preset conditions. If so, it is determined that the test result of the test case is a test pass. If not, it is determined that the test result of the test case is a test fail.

[0055] It should be noted that the solution type standard answer usually contains detailed information or explanatory descriptions. The solution type standard answer is not limited to specific keyword matching, but also needs to consider the overall semantic accuracy of the answer to ensure the rationality of the solution type standard answer. In some specific examples, the answer result text of the in-vehicle voice model is denoted as Ao Let the standard answer for the solution type be denoted as A r Let the semantic similarity be denoted as S, and set the preset condition as the preset similarity threshold

[0056] When the test question is of the solution explanation type, use the sentence-transformers algorithm to automatically calculate the answer result text A of the in-vehicle voice model o and the standard answer A for the solution type r for the semantic similarity S therebetween, and automatically determine whether the calculated semantic similarity S is greater than the preset similarity threshold. If the semantic similarity S is greater than the preset similarity threshold, it is determined that the test result of the test case is a pass; if the semantic similarity S is not greater than the preset similarity threshold, it is determined that the test result of the test case is a fail

[0057] In this embodiment, the standard answer for the solution explanation type question is set as the standard answer for the solution type, and when analyzing and comparing the answer result text with the standard answer for the solution type, first obtain the semantic similarity between the answer result text and the standard answer for the solution type, and then determine whether the semantic similarity meets the preset similarity threshold. Thus, the flexibility and reliability in testing solution explanation type questions can be effectively improved

[0058] In an embodiment of the present invention, when the question type of the test question is of the recommendation type, the standard answer is the information verification point set, the information verification point set includes multiple information verification points, and the information verification point is the attribute feature of the recommended target. Step S3 includes: generating the summary information of the answer result text; extracting the recommended items from the summary information; obtaining the detailed information set of the recommended items; determining whether the detailed information set is consistent with the information verification point set. If so, it is determined that the test result of the test case is a pass; if not, it is determined that the test result of the test case is a fail

[0059] In some specific examples, denote the information verification point set as E d The information verification point set E d contains multiple information verification points. For example, the number of information verification points can be three, four, five, six, or more than seven. For example, when the information verification point set E d is a recommended restaurant, then the multiple information verification points may include information such as the address, phone number, business hours of the restaurant

[0060] When the test question is of the recommendation type, first use the uer / t5-small-chinese-cluecorpussmall algorithm to automatically summarize the answer result text A of the in-vehicle voice model o extract the key information in the answer of the in-vehicle voice model to the test question, and generate the summary information A b . After that, determine the summary information Ab all the recommended items R mentioned, and from the summary information A b extract the recommended item R, and then use an online database to query and obtain the detailed information set R of the recommended item d .

[0061] After obtaining the detailed information set R of the recommended item d , check the detailed information set R of the recommended item one by one d to see if it is consistent with multiple information verification points in the information verification point set E d . If the detailed information set R of the recommended item d is consistent with multiple information verification points in the information verification point set E d , it is determined that the test result of the test case is a pass; if the detailed information set R of the recommended item d is inconsistent with multiple information verification points in the information verification point set E d , it is determined that the test result of the test case is a failure.

[0062] In this embodiment, the standard answer to the recommended type of question is set as the information verification point set. The information verification point set includes multiple information verification points. The information verification point is the attribute feature of the recommended target. And when analyzing the answer result text, first generate the summary information of the answer result text, then extract the recommended item from the summary information, then obtain the detailed information set of the recommended item, and then determine whether the detailed information set is consistent with the information verification point set. Thus, the answer result text of the in-vehicle voice model can be reliably verified, thereby effectively improving the accuracy of the test result.

[0063] In an embodiment of the present invention, when the test question is a music recommendation type question, multiple information verification points in the information verification point set include: singer, song name, music style, album and / or release year; when the test question is a movie recommendation type question, multiple information verification points in the information verification point set include: director, actor, release year, award-winning situation, score, type and / or running time; when the test question is a scenic spot recommendation type question, multiple information verification points in the information verification point set include: city, charge and / or scenic spot category.

[0064] When the test question is a music recommendation type question, for example, multiple information verification points in the information verification point set can be any one of singer, song name, music style, album and release year; another example is that multiple information verification points in the information verification point set can be any combination of singer, song name, music style, album and release year.

[0065] When the test question is a movie recommendation question, for example, multiple information verification points in the information verification point set can be any one of the director, actor, release year, award-winning situation, rating, genre, and running time; or, multiple information verification points in the information verification point set can be any combination of the director, actor, release year, award-winning situation, rating, genre, and running time.

[0066] When the test question is a scenic spot recommendation question, multiple information verification points in the information verification point set can be any one of the city, fee, and scenic spot category; or, multiple information verification points in the information verification point set can be any combination of the city, fee, and scenic spot category.

[0067] In some specific examples, when the test question is a music recommendation question, the information verification point set E d has multiple information verification points including: singer E s , song name E n , music style E st , album E a and release year E y1 ; when the test question is a movie recommendation question, the information verification point set E d has multiple information verification points including: director E co , actor E ac , release year E y2 , award-winning situation E aw , rating E r , genre E g and running time E l ; when the test question is a scenic spot recommendation question, the information verification point set E d has multiple information verification points including: city E ci , fee E ch and scenic spot category E t .

[0068] For music recommendation questions, the detailed information set R d of the recommendation item includes singer R s , song name R n , music style R st , album R a and release year R y1 , and the detailed information set R d of the recommendation item can be queried and obtained through an online music database; for movie recommendation questions, the detailed information set R d of the recommendation item includes director R co , actor R ac , release year R y2 , award-winning situation R aw , rating R r , genre R g and running time Rl , the detailed information set R of the recommended items d can be queried and obtained through an online movie database; for scenic spot recommendation problems, the detailed information set R of the recommended items d includes the city R ci , the charge R ch and the scenic spot category R t , the detailed information set R of the recommended items d can be queried and obtained through an online geographic information database and a tourist attraction database.

[0069] In this embodiment, multiple information verification points of the information verification point set for music recommendation problems are set as the singer, song name, music genre, album, and / or release year; multiple information verification points of the information verification point set for movie recommendation problems are set as the director, actor, release year, award-winning situation, rating, type, and / or film length; multiple information verification points of the information verification point set for scenic spot recommendation problems are set as the city, charge, and / or scenic spot category, which can effectively improve the comprehensiveness of verifying the answer text of the in-vehicle voice model, thereby effectively improving the reliability of the test results.

[0070] In an embodiment of the present invention, when the problem type of the test problem is vehicle control, the standard answer is the execution result checkpoint set, and the execution result checkpoint set includes multiple execution result state parameters. The execution result state parameter is the target state parameter after the vehicle executes the test problem. Step S3 includes: obtaining the state parameter set of the control node corresponding to the test problem through the CAN bus; determining whether the state parameter set is consistent with the execution result checkpoint set; if so, determining that the test result of the test case is a pass, and if not, determining that the test result of the test case is a fail.

[0071] It should be noted that the CAN signal in an automobile refers to the data signal transmitted by the Controller Area Network (abbreviated as CAN) technology. CAN is a protocol for in-vehicle communication. The CAN bus can connect multiple electronic control units (ECUs) in the automobile, thereby realizing real-time data sharing.

[0072] In some specific examples, when the test problem is vehicle control, the state parameter set A of the control node corresponding to the test problem can be obtained through the CAN bus cp , including the air conditioner switch, air conditioner wind speed, air conditioner set temperature, air conditioner circulation mode, window state, seat position, and the switch states of each setting item, etc. Then, analyze and determine the state parameter set A cp and the execution result checkpoint set E cp whether they are consistent. If the state parameter set A cp and the execution result checkpoint set Ecp If they are consistent, it is determined that the test result of the test case passes the test; if the set of status parameters A cp is inconsistent with the execution result checkpoint set E cp it is determined that the test result of the test case fails the test.

[0073] In this embodiment, the standard answers to vehicle control problems are set as the execution result checkpoint set, and the execution result checkpoint set includes multiple execution result status parameters. The execution result status parameters are the target status parameters after the vehicle executes the test problems. When analyzing the answer result text, first obtain the set of status parameters of the vehicle corresponding to the test problem through the CAN bus, and then determine whether the set of status parameters is consistent with the execution result checkpoint set, which can effectively improve the test efficiency and effectively ensure the accuracy of the test result.

[0074] In an embodiment of the present invention, before step S2, the test method further includes: retrieving whether the answer result text contains keywords corresponding to the test problem; if so, execute step S2, if not, it is determined that the test result of the test case fails the test.

[0075] It should be noted that each test problem has a corresponding preset keyword. The keyword is usually the core vocabulary directly related to the problem, and the number of keywords can be one or more. When the number of keywords is multiple, for example, the number of keywords can be two, three, four, five, six or more.

[0076] In some specific examples, before confirming that the answer result text contains keywords corresponding to the test problem, it is necessary to perform necessary preprocessing on the answer result text of the in-vehicle voice model. For example, it is necessary to pre-remove punctuation marks from the answer result text. Performing preprocessing on the answer result text can more accurately match keywords, thereby improving the test efficiency. Further, when confirming that the answer result text contains keywords corresponding to the test problem, methods such as exact matching, fuzzy matching, and semantic analysis can be used to further improve the accuracy of the matching.

[0077] In some specific examples, the keyword corresponding to the test problem is denoted as K. After using adb logcat to extract the answer result text A o then, perform a search for the keyword K corresponding to the test problem on the answer result text A o If the keyword K is retrieved, then confirm the problem type corresponding to the test problem, then confirm the analysis and determination mode corresponding to the problem type, and then perform further analysis and determination; if the keyword K is not detected, it is determined that the test result of the test case fails the test.

[0078] In this embodiment, by retrieving whether the answer result text contains keywords corresponding to the test question, the answer result text of the in-vehicle voice model can be screened in advance, so as to ensure that the answer result text of the in-vehicle voice model is relevant to the test question and not overly abstract, thereby effectively improving the test efficiency.

[0079] In an embodiment of the present invention, the test case set includes multiple test cases. After step S3, the test method further includes: S4, repeating steps S1 - S3 until all test cases are traversed.

[0080] In some specific examples, when a test case in the test case set is tested, the next untested test case is automatically selected from the test case set until all test cases in the test case set are tested. That is to say, for each test case in the test case set, the foregoing steps S1 - S3 are used for testing, so that the test results corresponding to each test case in the test case set can be obtained, and thus the automated test of the in-vehicle voice model can be effectively realized.

[0081] S5, analyze all test results to obtain the test conclusion of the in-vehicle voice model.

[0082] In some specific examples, after obtaining the test results corresponding to each test case in the test case set, all test results are summarized and comprehensively analyzed, and the in-vehicle voice model is overall evaluated according to the analysis results, pointing out the strengths and weaknesses of the current in-vehicle voice model. At the same time, corresponding improvement measures or suggestions are proposed according to the discovered problems, so as to continuously improve the in-vehicle voice model.

[0083] In this embodiment, after step S3, steps S1 - S3 are repeated until all test cases are traversed, and then all test results are analyzed to obtain the test conclusion of the in-vehicle voice model, which can not only ensure that all test cases in the test case set can be tested, but also effectively improve the automation level of the test.

[0084] In an embodiment of the present invention, step S5 includes: calculating the test pass rate of multiple test cases in the test case set; when the test pass rate is greater than or equal to the preset pass rate, the in-vehicle voice model passes the test.

[0085] In some specific examples, first count the number of test cases that pass the test among all test cases, and then use the following formula to calculate the test pass rate of multiple test cases in the test case set: Test pass rate = (Number of test cases that pass the test / Total number of test cases) × 100%.

[0086] According to project requirements and quality standards, a preset passing rate is pre-set as the threshold for determining whether the in-vehicle voice model successfully passes the test. When the test passing rate is less than the preset passing rate, the in-vehicle voice model fails the test; when the test passing rate is greater than or equal to the preset passing rate, the in-vehicle voice model passes the test.

[0087] In this embodiment, by calculating the test passing rates of multiple test cases in the test case set and comparing the test passing rate with the preset passing rate to analyze and determine whether the in-vehicle voice model passes the test, it can provide a clear standard for measuring the overall performance of the in-vehicle voice model, thus facilitating the analysis and determination of whether the in-vehicle voice model can pass the test, and further contributing to subsequent improvement work.

[0088] According to the test device 100 of the in-vehicle voice model according to the second aspect embodiment of the present invention, the test device 100 includes a broadcast module 10, an acquisition module 20, a confirmation module 30, an analysis and comparison module 40, and a determination module 50.

[0089] The broadcast module 10 is used to broadcast the test questions of the test cases in the test case set; the acquisition module 20 is used to acquire the answer result text of the in-vehicle voice model; the confirmation module 30 is used to confirm the question type corresponding to the test question and the analysis and determination mode corresponding to the question type; the analysis and comparison module 40 is used to analyze and compare the answer result text with the standard answer of the test question; the determination module 50 is used to judge the test result of the test case and to judge the test conclusion of the in-vehicle voice model.

[0090] The main responsibility of the broadcast module 10 is to play each test question in the test case set in the form of voice. In some specific examples, an artificial mouth is set in the broadcast module 10. It should be noted that the artificial mouth is a voice broadcast device (loudspeaker) with high precision and good acoustic properties. When testing the in-vehicle voice model, the artificial mouth is set in the real vehicle to simulate a user asking questions to the in-vehicle voice model. That is to say, the artificial mouth can restore the real scenario when a user asks questions to the in-vehicle voice model. Thus, it can effectively save labor costs.

[0091] The acquisition module 20 is used to receive and record the answer result of the in-vehicle voice model to the test questions sent by the broadcast module 10. In some specific examples, the acquisition module 20 is adb logcat. After the in-vehicle voice model answers the test questions proposed by the artificial mouth, the in-vehicle infotainment system can record and store the answer result text of the in-vehicle voice model in the log information, and adb logcat can access, retrieve, and extract the answer result text of the in-vehicle voice model from the log information.

[0092] It should be noted that adb logcat is part of the Android Debug Bridge (ADB) tool and can be used to automatically view the log information of the connected device. That is to say, using adb logcat can automatically extract the response result text of the in-vehicle voice model, enabling the systematic collection of test results each time without manual intervention. Thus, the automation level of the test method for the in-vehicle voice model can be effectively improved.

[0093] The task of the confirmation module 30 is to determine the type of the test question and select an appropriate analysis and judgment mode accordingly. Different types of questions require different evaluation criteria and methods. In some specific examples, the confirmation module 30 automatically classifies the test questions based on predefined rules or machine learning algorithms to accurately and quickly confirm the type of the test question.

[0094] The analysis and comparison module 40 is responsible for comparing the response result text with the standard answer of the test question to evaluate whether the answer of the in-vehicle voice model is accurate. For different types of questions, the analysis and comparison module 40 uses different logics and methods for analysis. The task of the determination module 50 is to make a judgment on the result of the test question of a single test case and finally decide whether the entire in-vehicle voice model passes the test. The determination module 50 can not only judge the success or failure of a single test case but also draw an overall conclusion based on the performance of all test cases.

[0095] According to the test device 100 of the in-vehicle voice model of the second aspect of the present invention, by setting the broadcast module 10, the acquisition module 20, the confirmation module 30, the analysis and comparison module 40, and the determination module 50 in the test device 100, the broadcast module 10 is used to broadcast the test questions of the test cases in the test case set, the acquisition module 20 is used to acquire the response result text of the in-vehicle voice model, the confirmation module 30 is used to confirm the type of the test question and the corresponding analysis and judgment mode, the analysis and comparison module 40 is used to analyze and compare the response result text with the standard answer of the test question, and the determination module 50 is used to judge the test result of the test case and the test conclusion of the in-vehicle voice model. It can not only realize the automated test of the in-vehicle voice model but also effectively improve the test efficiency of the in-vehicle voice model.

[0096] According to the test terminal 600 of the third aspect embodiment of the present invention, the test terminal 600 includes a memory, a processor 70, and a test program 612 of the in-vehicle voice model stored in the memory and executable on the processor 70. When the test program 612 of the in-vehicle voice model is executed by the processor 70, it implements the steps of the test method of the in-vehicle voice model.

[0097] In some specific examples, the test terminal 600 includes a processor 70 and a memory connected by a system bus 700. Among them, the processor 70 of the test terminal 600 is used to provide computing and control capabilities. The memory of the test terminal 600 includes a non-volatile storage medium 61 and an internal memory 62. The non-volatile storage medium 61 stores an operating system 611 and a test program 612 of the in-vehicle voice model. The internal memory 62 provides an environment for the operation of the operating system 611 and the test program 612 of the in-vehicle voice model in the non-volatile storage medium 61. When the test program 612 of the in-vehicle voice model is executed by the processor 70, the steps of any one of the above-mentioned test programs 612 of the in-vehicle voice model are implemented.

[0098] According to the test terminal 600 of the third aspect of the present invention, by setting a memory, a processor 70 and a test program 612 of the in-vehicle voice model stored on the memory and executable on the processor 70 in the test terminal 600, when the test program 612 of the in-vehicle voice model is executed by the processor 70, the steps of the test method of the in-vehicle voice model are implemented. It can not only realize the automated test of the in-vehicle voice model, but also effectively improve the test efficiency of the in-vehicle voice model.

[0099] According to the computer-readable storage medium of the embodiment of the fourth aspect of the present invention, a test program of the in-vehicle voice model is stored on the computer-readable storage medium. When the test program of the in-vehicle voice model is executed by a processor, the steps of the test method of the in-vehicle voice model are implemented.

[0100] According to the computer-readable storage medium of the fourth aspect of the present invention, by storing a test program of the in-vehicle voice model on the computer-readable storage medium, when the test program of the in-vehicle voice model is executed by a processor, the steps of the test method of the in-vehicle voice model according to the first aspect of the present invention are implemented. It can not only realize the automated test of the in-vehicle voice model, but also effectively improve the test efficiency of the in-vehicle voice model.

[0101] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention.

[0102] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more unless otherwise specifically defined.

[0103] In the present invention, unless otherwise clearly specified and defined, terms such as "mounted", "connected", "coupled", "fixed", etc. shall be construed in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection, an electrical connection, or a communication connection; it may be directly connected, or indirectly connected through an intermediate medium, and it may be the internal communication of two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0104] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0105] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for testing a vehicle-mounted speech model, characterized in that: include: S1, broadcasting the test questions of the test cases in the test case set, and obtaining the answer result text of the in-vehicle voice model; S2, confirming the question type corresponding to the test question, and confirming the analysis and determination mode corresponding to the question type; S3, according to the analysis and determination mode, analyzing and comparing the answer result text with the standard answer to the test question to obtain the test result of the test case.

2. The method for testing a vehicle-mounted speech model according to claim 1, characterized in that: The question types include: answer explanation type, recommendation type and vehicle control type.

3. The method for testing the vehicle-mounted speech model according to claim 2, characterized in that: When the question type of the test question is a solution explanation type, the standard answer is a solution type standard answer, and step S3 includes: Comparing the answer result text with the answer type standard answer to obtain the semantic similarity between the answer result text and the answer type standard answer; Determine whether the semantic similarity meets the preset conditions, If yes, the test result of the test case is determined to be passed. If not, the test result of the test case is determined to be test failure.

4. The method for testing a vehicle-mounted speech model according to claim 2, characterized in that: When the question type of the test question is a recommendation type, the standard answer is an information verification point set, the information verification point set includes a plurality of information verification points, and the information verification points are attribute features of the recommended target. Step S5 includes: Generate summary information of the answer result text; extracting recommendation items from the summary information; Obtain detailed information set of the recommended item; Determine whether the detailed information set is consistent with the information verification point set, If yes, the test result of the test case is determined to be passed. If not, the test result of the test case is determined to be test failure.

5. The method for testing the vehicle-mounted speech model according to claim 4, characterized in that: When the test question is a music recommendation question, the multiple information verification points of the information verification point set include: singer, song title, music style, album and / or release year; When the test question is a movie recommendation question, the multiple information verification points of the information verification point set include: director, actor, release year, awards, rating, genre and / or length; When the test question is a question of scenic spot recommendation, the multiple information verification points of the information verification point set include: city, fee and / or scenic spot category.

6. The method for testing a vehicle-mounted speech model according to claim 2, characterized in that: When the question type of the test question is vehicle control, the standard answer is an execution result checkpoint set, and the execution result checkpoint set includes a plurality of execution result state parameters, and the execution result state parameters are target state parameters after the vehicle executes the test question. Step S5 includes: Acquire a set of state parameters of a control node corresponding to the test problem through a CAN bus; Determining whether the state parameter set is consistent with the execution result checkpoint set; If yes, the test result of the test case is determined to be passed. If not, the test result of the test case is determined to be test failure.

7. The method for testing a vehicle-mounted speech model according to any one of claims 1 to 6, characterized in that: Before step S2, the testing method further includes: Searching whether the answer result text contains keywords corresponding to the test question; If yes, execute step S2, if no, determine that the test result of the test case is test failure.

8. The method for testing a vehicle-mounted speech model according to any one of claims 1 to 6, characterized in that: The test case set includes a plurality of test cases. After step S3, the test method further includes: S4, repeating steps S1-S3 until the test cases are traversed; S5, analyzing all the test results to obtain the test conclusion of the vehicle-mounted speech model.

9. The method for testing a vehicle-mounted speech model according to claim 8, characterized in that: Step S5 includes: Calculating the test pass rates of the multiple test cases in the test case set; When the test pass rate is greater than or equal to a preset pass rate, the in-vehicle voice model passes the test.

10. A vehicle-mounted speech model testing device, characterized in that: include: A reporting module, used to report test problems of test cases in a test case set; An acquisition module, used to obtain the answer result text of the vehicle-mounted voice model; A confirmation module, used to confirm the question type corresponding to the test question and the analysis and determination mode corresponding to the question type; An analysis and comparison module, used for analyzing and comparing the answer result text with the standard answer to the test question; A determination module is used to determine the test result of the test case and to determine the test conclusion of the vehicle-mounted speech model.

11. A test terminal, characterized in that: The test terminal includes a memory, a processor, and a test program for a vehicle-mounted speech model stored in the memory and executable on the processor. When the test program for the vehicle-mounted speech model is executed by the processor, the steps of the test method for the vehicle-mounted speech model as described in any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a test program for the vehicle-mounted speech model, and when the test program for the vehicle-mounted speech model is executed by the processor, the steps of the test method for the vehicle-mounted speech model as described in any one of claims 1 to 9 are implemented.

Images