Vehicle-mounted voice test method and device, electronic equipment and readable storage medium

Abstract

The invention provides a vehicle-mounted voice test method and device, electronic equipment and a readable storage medium. The method comprises the following steps: acquiring target corpus data; obtaining a trained test corpus generation model, and performing generalization corpus generation based on the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data; generating a test audio according to the test corpus data, and playing the test audio, so that the vehicle-mounted voice system collects the test audio; and obtaining feedback data generated by the vehicle-mounted voice system based on the test audio, and obtaining a test result according to the test demand data and the feedback data. The generalized test corpus data is generated based on the target corpus data through the trained test corpus generation model, so that the test corpus data can cover more diversified scenes, the problem of corpus limitation is solved, then audio testing is performed based on the test corpus, and the test accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle testing, and in particular to a vehicle-mounted voice testing method, device, electronic equipment and readable storage medium. Background Art

[0002] In the prior art, when performing voice testing on a vehicle, the test is often achieved by playing voice data in a voice library; however, in actual use of the vehicle, since the user's speaking style and specific speaking content are not fixed, the existing vehicle voice testing method is limited by the limitations of voice data and is difficult to meet the requirements of different usage scenarios, and the test results are unsatisfactory. Summary of the invention

[0003] The main purpose of the present invention is to propose a vehicle-mounted voice test method, device, electronic device and readable storage medium, aiming to solve the problem in the prior art that the vehicle-mounted voice test results are difficult to meet the requirements of different usage scenarios and the test results are not ideal.

[0004] To achieve the above object, the present invention provides a vehicle-mounted voice testing method, the method comprising the steps of:

[0005] Acquire test requirement data, and determine target corpus data in the test requirement data;

[0006] Acquire a trained test corpus generation model, and perform generalized corpus generation on the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the trained test corpus generation model is trained using generalized corpus data as samples, and the amount of the test corpus data is greater than the amount of the target corpus data;

[0007] Generating a test audio according to the test corpus data, and playing the test audio so that the vehicle-mounted voice system can collect the test audio;

[0008] Acquire feedback data generated by the in-vehicle voice system based on the test audio, and obtain a test result according to the test requirement data and the feedback data.

[0009] Optionally, the step of obtaining the trained test corpus generation model includes:

[0010] Obtain the trained corpus generalization model and basic corpus data;

[0011] Generalizing the basic corpus data through the trained corpus generalization model to obtain the generalized corpus data corresponding to the basic corpus data;

[0012] An initial test corpus generation model is obtained, and the initial test corpus generation model is trained using generalized corpus data as a sample to obtain the trained test corpus generation model.

[0013] Optionally, the test corpus generation model completed through the training performs generalized corpus generation on the target corpus data to obtain test corpus data corresponding to the target corpus data, including:

[0014] Obtaining target corpus configuration data in the test requirement data, and using the target corpus configuration data as operation configuration data of the trained test corpus generation model, wherein the target corpus configuration data includes target corpus complexity and target corpus quantity;

[0015] The target corpus data is generalized and generated by the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the corpus complexity of the test corpus data is the target corpus complexity, and the number of the test corpus data is the target corpus number.

[0016] Optionally, generating a test audio according to the test corpus data includes:

[0017] Acquire target language feature data in the test requirement data;

[0018] Matching a digital person corresponding to the target language feature data;

[0019] The test corpus data is sent to the digital human so that the digital human plays the test corpus data to generate the test audio.

[0020] Optionally, the obtaining feedback data generated by the in-vehicle voice system based on the test audio, and obtaining a test result according to the test requirement data and the feedback data includes:

[0021] Determining whether the test audio data is of a question-and-answer type;

[0022] If the test audio data is of question and answer type, collecting the reply content played by the in-vehicle voice system based on the test audio;

[0023] Determining whether the reply content matches the test audio data;

[0024] If the reply content matches the test audio data, the test result is passed.

[0025] Optionally, the determining whether the reply content matches the test audio data includes:

[0026] Acquire multiple large models, and input the test audio and the reply content into each of the large models respectively, so that the large model determines whether the reply content is the answer corresponding to the test audio, and obtains multiple matching results;

[0027] Calculate the correct rate corresponding to the matching result, and determine whether the correct rate is greater than a preset correct threshold;

[0028] If the accuracy rate is greater than the preset accuracy threshold, the reply content matches the test audio data.

[0029] Optionally, obtaining the test result according to the test requirement data and the feedback data includes:

[0030] Receiving a regression test instruction, and determining regression corpus data corresponding to the regression test instruction in the test corpus data;

[0031] Storing the regression corpus data in a regression corpus;

[0032] If the regression test is triggered, the regression corpus data is used as the target corpus data for testing;

[0033] After the test is completed, the regression corpus data is deleted from the regression corpus.

[0034] To achieve the above object, the present invention further provides a vehicle-mounted voice test device, the vehicle-mounted voice test device comprising:

[0035] A first acquisition module is used to acquire test requirement data and determine target corpus data in the test requirement data;

[0036] A second acquisition module is used to acquire a trained test corpus generation model, and perform generalized corpus generation on the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the trained test corpus generation model is trained with generalized corpus data as samples, and the amount of the test corpus data is greater than the amount of the target corpus data;

[0037] A first generating module, configured to generate a test audio according to the test corpus data, and play the test audio so that the vehicle voice system can collect the test audio;

[0038] The third acquisition module is used for the feedback data generated by the in-vehicle voice system based on the test audio, and obtains the test result according to the test requirement data and the feedback data.

[0039] To achieve the above objectives, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of the in-vehicle voice testing method as described above when executed by the processor.

[0040] To achieve the above objectives, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the in-vehicle voice testing method as described above are implemented.

[0041] The present invention proposes an in-vehicle voice testing method, device, electronic device and readable storage medium, which obtain test requirement data and determine target corpus data in the test requirement data; obtain a trained test corpus generation model, and generate generalized corpus for the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the trained test corpus generation model is trained with generalized corpus data as samples, and the number of test corpus data is greater than the number of target corpus data; generate test audio according to the test corpus data, and play the test audio so that the in-vehicle voice system collects the test audio; obtain feedback data generated by the in-vehicle voice system based on the test audio, and obtain a test result according to the test requirement data and the feedback data. By generating generalized test corpus data based on the target corpus data through the trained test corpus generation model, the test corpus data can cover more diverse scenarios, solving the problem of corpus limitations. Then, audio testing can be performed based on the test corpus, which can improve the accuracy of the test, expand the depth and breadth of the test, and improve the reliability of the in-vehicle voice system. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 It is a flowchart of the first embodiment of the vehicle voice testing method of the present invention;

[0045] Figure 2It is a schematic diagram of the principle of the corpus generalization model in the vehicle-mounted speech testing method of the present invention;

[0046] Figure 3 It is a schematic diagram of the principle of testing the prediction generation model in the vehicle-mounted speech testing method of the present invention;

[0047] Figure 4 It is a schematic diagram of the application process of the digital twin in the vehicle-mounted voice testing method of the present invention;

[0048] Figure 5 A schematic diagram of creating a digital avatar in the vehicle-mounted voice testing method of the present invention;

[0049] Figure 6 A schematic diagram of generating ambient sound in the vehicle-mounted voice testing method of the present invention;

[0050] Figure 7 A schematic diagram of the accuracy calculation in the vehicle-mounted speech testing method of the present invention;

[0051] Figure 8 It is a schematic diagram of the module structure of the electronic device of the present invention. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present application.

[0053] The present invention provides a vehicle-mounted voice testing method, referring to Figure 1 , Figure 1 This is a flow chart of a first embodiment of a vehicle voice testing method according to the present invention, wherein the method comprises the following steps:

[0054] Step S10, obtaining test requirement data, and determining target corpus data in the test requirement data;

[0055] The test requirement data is used to indicate the requirements for voice testing. For a specific voice test, the specific content and method of the test vary depending on the vehicle and purpose of the test. Therefore, the user can clearly define the test requirements by setting the test requirement data.

[0056] The test requirement data includes target corpus data; it can be understood that in voice testing, specific audio needs to be emitted so that the on-board voice system can collect and process the audio and then perform related operations; and the target corpus data indicates the content of the audio. The target corpus data includes one or more specific corpora. The target corpus data can be input by the user himself or specified by the user in the corpus.

[0057] Step S20, obtaining a trained test corpus generation model, and performing generalized corpus generation on the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the trained test corpus generation model is trained with the generalized corpus data as samples, and the amount of the test corpus data is greater than the amount of the target corpus data;

[0058] The test corpus generation model is used to generalize the target corpus data to obtain the test corpus data; it is understandable that the target corpus data indicates the specific corpus content, but the same meaning can often be expressed by different words, such as "turn on the seat heating" can also be expressed as "turn on the seat heating", the two have the same meaning, but there are differences in the text content. Therefore, if the test is based on a single corpus, it is easy to ignore the recognition of other similar words by the in-vehicle voice system; therefore, in this embodiment, a test corpus generation model is set, and the target corpus data is generalized by the trained test corpus generation model to generate a corpus to obtain the test corpus data; it is understandable that the test corpus data is generalized from the target corpus data, specifically, the test corpus data is the same or similar in meaning to the target corpus data, and at the same time, the number of test corpus data is greater than the number of target corpus data, which realizes the expansion of the corpus data, so that the test corpus data can cover more diverse scenarios.

[0059] The generalized corpus data is used to indicate the generalization process. The test corpus generation model is trained through the generalized corpus data so that the test corpus generation model can have the ability to generalize the corpus, thereby being able to generate generalized corpus for the target corpus data.

[0060] Step S30, generating a test audio according to the test corpus data, and playing the test audio so that the vehicle voice system can collect the test audio;

[0061] Step S40, obtaining feedback data generated by the vehicle-mounted voice system based on the test audio, and obtaining a test result according to the test requirement data and the feedback data.

[0062] After obtaining the test corpus data, a test audio can be generated according to the test corpus data and played; the in-vehicle voice system collects the test audio, processes the test audio, and then performs corresponding operations based on the test audio content to generate feedback data; it can be understood that based on the specific type of the test audio, the in-vehicle voice system performs different operations and the types of feedback data generated are also different; if the test data involves the execution of an action, it can be determined that the corresponding action is vehicle computer control or vehicle control, and the corresponding feedback data is obtained based on specific related interface changes and vehicle signals; specifically, if the test audio indicates that the air conditioner is turned on, the operation performed by the in-vehicle voice system is to turn on the air conditioner, and the feedback data generated is the status of the air conditioner; if the test audio indicates that the seat is heated, the operation performed by the in-vehicle voice system is to turn on the seat heating, and the feedback data generated is the status of the seat; for example, if the test audio is a question, the operation performed by the in-vehicle voice system is to answer the question, and the feedback data generated is the answer audio of the in-vehicle voice system.

[0063] It can be understood that the test requirement data indicates the state of the expected execution of the in-vehicle voice system, while the feedback data indicates the state of the actual execution of the in-vehicle voice system. Therefore, the test result can be determined by comparing the test requirement data with the feedback data; specifically, when the feedback data is consistent with that indicated in the test requirement data, the test result is passed; when the feedback data is inconsistent with that indicated in the test requirement data, the test result is failed.

[0064] This embodiment generates generalized test corpus data based on the target corpus data through the trained test corpus generation model, so that the test corpus data can cover more diverse scenarios, solving the problem of corpus limitations, and then performing audio testing based on the test corpus, so that the accuracy of the test can be improved, the depth and breadth of the test can be expanded, and the reliability of the in-vehicle voice system can be improved.

[0065] Further, see Figure 2 and Figure 3 In the second embodiment of the vehicle voice test method of the present invention proposed based on the first embodiment of the present invention, the step before step S20 includes the following steps:

[0066] Step S50, obtaining the trained corpus generalization model and basic corpus data;

[0067] Step S60, generalizing the basic corpus data by using the trained corpus generalization model to obtain the generalized corpus data corresponding to the basic corpus data;

[0068] Step S70, obtaining an initial test corpus generation model, and training the initial test corpus generation model using generalized corpus data as samples to obtain the trained test corpus generation model.

[0069] The corpus generalization model is used to generate generalized corpus data; it can be understood that the generalized corpus data indicates the generalization process, that is, it indicates the expression method for the specific meaning; and for different users and different scenarios, the expression method is different. Therefore, in this embodiment, the generalized corpus data is generated by training the corpus generalization model, thereby generating a variety of generalized corpus data to cover a variety of usage scenarios; the generated generalized corpus data can be displayed to the user so that the user can add, delete, modify, check and set permissions for the generalized corpus data.

[0070] The training method of the corpus generalization model can be set based on actual needs, such as collecting some high-frequency corpora through data embedding. For example, when collecting data, if what the user says meets the data embedding trigger condition, the user's words are collected to obtain the embedding data. The specific type of embedding data required can be set based on actual needs. After obtaining the embedding data, the embedding data can be compared with the corpus data in the corpus. If the corpus does not contain the same corpus data as the embedding data, the embedding data is supplemented with fixed reply words, execution actions and other related fields, and stored in a relational database. When it is necessary to train the corpus generalization model, the embedding data is called for training.

[0071] The generalized corpus data can be obtained by generalizing the basic corpus data; after obtaining the generalized corpus data, the initial test corpus generation model can be trained by the generalized corpus data to obtain the trained test corpus generation model.

[0072] Further, in a third embodiment of the vehicle-mounted voice testing method of the present invention proposed based on the first embodiment of the present invention, step S20 includes the steps of:

[0073] Step S21, obtaining target corpus configuration data in the test requirement data, and using the target corpus configuration data as operation configuration data of the trained test corpus generation model, wherein the target corpus configuration data includes target corpus complexity and target corpus quantity;

[0074] Step S22, generalizing the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the corpus complexity of the test corpus data is the target corpus complexity, and the number of the test corpus data is the number of the target corpus.

[0075] The target corpus configuration data is used to indicate the situation of the generated test corpus data; it is understood that the requirements for the test corpus data are different based on different test needs; specifically, in this embodiment, the target corpus configuration data including the corpus complexity and the corpus quantity is set, and the user can set the target corpus complexity and the target corpus quantity in the test requirement data; it is understood that the corpus complexity is used to indicate the complexity of the generated training corpus data, and the complexity can include the number of characters, the number of words, the order of parts of speech, etc.; the higher the corpus complexity, the higher the requirement for the recognition ability of the vehicle voice system, and the lower the corpus complexity, the lower the requirement for the recognition ability of the vehicle voice system. The corpus quantity is used to indicate the number of generated training corpus data. Specifically, in this embodiment, the ratio between the target corpus data and the training corpus data is set at one to N, and N is the number of target corpus, that is, each target corpus data is generalized to generate 10 training corpus data; for example, when the number of target corpus is 10, the number of training corpus data generated by the test corpus generation model is 10 times the target corpus data.

[0076] In this embodiment, by setting the corpus complexity and the corpus quantity, the user can set the generation of test corpus data based on actual test needs.

[0077] In other embodiments, more parameters can be set in the target corpus configuration data, such as whether to give priority to matching the corpus in the generalized corpus, whether to retain the basic corpus, and whether to append all corpora with the same semantics, version, and module in the generalized corpus; the generalized corpus is a library for storing the corpus data generated by generalization; when the user sets the corpus in the generalized corpus to be matched first, the generalized corpus data with the same meaning as the target corpus data is matched in the generalized corpus as the test corpus data. When the number of generalized corpus data contained in the generalized corpus cannot meet the number of target corpuses, the insufficient test corpus data is generated by the trained test corpus generation model.

[0078] See also Figure 4 , you can specifically set up the requirements document management system and the requirements document learning system; the requirements document management is divided into two dimensions: test version and test module. The test version and test module are created by system users. Users can upload the corresponding test requirement data for analysis and learning by the downstream requirements document learning system, so that the model in the requirements document learning system can generate basic corpus data; the created versions and modules are provided to users through the interface for selection. The management of requirements documents is implemented through a relational database, and the system generates a requirements management table to facilitate data maintenance and management.

[0079] The requirement document learning system is based on a large model dedicated to the training of standardized requirement documents. The model is responsible for extracting regular test corpus, fixed response words, and execution actions in the document, and marking relevant attributes, such as execution action tags, response word tags, vehicle computer or vehicle control tags, and open response word tags. In addition, the model also processes requirement version and module information. The output structured data is stored in a relational database. The system also needs to process knowledge question and answer requirements based on the large model, incorporate question and answer categories into the basic corpus data, and mark the corresponding fields in the corpus as AI question and answer categories.

[0080] Further, see Figure 3 In the fourth embodiment of the vehicle voice testing method of the present invention proposed based on the first embodiment of the present invention, the step S30 includes the steps of:

[0081] Step S31, obtaining target language feature data in the test requirement data;

[0082] Step S32, matching a digital person corresponding to the target language feature data;

[0083] Step S33: sending the test corpus data to the digital human so that the digital human plays the test corpus data to generate the test audio.

[0084] The target language feature data is used to indicate pronunciation characteristics; it is understandable that different users have different pronunciation methods, and the target language feature data includes but is not limited to settings for region, dialect type, age, gender and other features; therefore, during the test process, specific user groups can be set for testing.

[0085] In this embodiment, a digital person is set; the digital person is a virtual system capable of speech generation; after the target language feature data is determined, it is necessary to generate a test audio corresponding to the target language feature data; specifically, a corresponding digital person can be set for different target language feature data, and after the target language feature data is determined, the digital person corresponding to the target language feature data can be matched, and then the test corpus data is played through the artificial mouth of the digital person to obtain the test audio; in other embodiments, an initial digital person can also be set, and after the target language feature data is determined, the configuration of the initial digital person is set to the target language feature data, so that the digital person has the audio generation capability of the target language feature data, and then the test corpus data is played through the digital person to obtain the test audio. It can be understood that when the test audio is played for the first time, the digital person is called, and when the subsequent continuous test audio is played, it is played directly through the digital person without repeated calls.

[0086] The specific setting method of digital human can be set based on actual needs; users can also create digital human by themselves, see Figure 5 , as a digital avatar of oneself, the user can input his or her own information and speech audio according to needs, so as to build a digital avatar corresponding to the user's speaking style.

[0087] It is understandable that, in the prior art, the test corpus is usually read aloud by the test personnel to provide the test audio; and in order to realize the test of speech with different pronunciation characteristics, in the prior art, it is necessary to collect audio of users with different pronunciation characteristics and then play it. However, this method is limited by the samples. On the one hand, it is necessary to collect a large number of audios with different pronunciation characteristics in advance. On the other hand, for some pronunciation characteristics, there may be a problem of less samples; therefore, in this embodiment, a digital human is set, and the corresponding digital human is set or selected through different target language feature data, so that speech with different pronunciation characteristics can be played through the digital human; thereby, it is not limited by the samples. For the same test corpus data, the digital human corresponding to different target language features can be played with different pronunciation characteristics, so that the sample size is expanded at the level of pronunciation characteristics; so that the speech test is not limited by the specific test personnel or the source of the recognition sample.

[0088] In specific applications, different digital humans can be used to broadcast any corpus data, without having to rely on the traditional method of recording audio of fixed corpus; after setting up digital human technology, the corpus can be expanded based on actual needs, solving the problem of sound source limitations, and at the same time solving the problem of high time costs required to collect different corpus and different sound characteristics through the traditional method of recording fixed audio. In the existing technology, each time a different corpus is added, it is necessary to find a corresponding person to re-record, while digital human technology can be used at any time after training is completed, greatly increasing the flexibility of adding new corpus.

[0089] Furthermore, the specific playback mode of the test corpus data can also be set, such as setting the number of playbacks for each target corpus data, and the test corpus data generated by the target corpus data will be played with the corresponding number of playbacks when played; another example is setting the playback parameters of the digital human, such as speaking speed, number of repetitions, and interval duration.

[0090] Furthermore, the ambient sound can be played simultaneously with the test corpus data to simulate the speech scene; see Figure 6For example, the system can collect various different environmental sounds and provide a user interface for users to filter and listen by category to determine which environmental sound to use for subsequent testing; the system has a built-in large model to learn the environmental sounds of different real scenes, and can generate different environmental sounds according to the different needs of users. The generated environmental sounds are managed through a relational database and an interface is provided for the test execution system to use. Users can set the required environmental sound type, scene and volume in the test requirement data. When the test corpus data needs to be played, the corresponding environmental sound can be extracted and played at the same time.

[0091] In this embodiment, by setting a digital human and setting the digital human's speaking mode through target language feature data, the problem of being limited to the tester's own voice source can be avoided and the test scenario can be expanded.

[0092] Further, in a fifth embodiment of the vehicle-mounted voice testing method of the present invention proposed based on the first embodiment of the present invention, step S40 includes the steps of:

[0093] Step S41, determining whether the test audio data is of question-answer type;

[0094] Step S42, if the test audio data is of question and answer type, collecting the reply content played by the in-vehicle voice system based on the test audio;

[0095] Step S43, determining whether the reply content matches the test audio data;

[0096] Step S44: if the reply content matches the test audio data, the test result is passed.

[0097] When the test audio data is of the question-and-answer type, the test scenario is the playback of the test audio data. The in-vehicle voice system recognizes the test audio data, obtains the question corresponding to the test audio data, searches for the answer to the question, and then plays the answer to the question. The played answer to the question is the reply content.

[0098] In this case, it is necessary to verify whether the reply content matches the test audio data, specifically whether it matches the question corresponding to the test audio data. If it matches, it is considered that the in-vehicle voice system responds correctly, so the test passes. If it does not match, it is considered that the in-vehicle voice system does not respond correctly, so the test fails.

[0099] Further, see Figure 7 , the step S43 comprises the steps of:

[0100] Step S431, obtaining multiple large models, and inputting the test audio and the reply content into each of the large models respectively, so that the large model determines whether the reply content is the answer corresponding to the test audio, and obtains multiple matching results;

[0101] Step S432, calculating the accuracy rate corresponding to the matching result, and determining whether the accuracy rate is greater than a preset accuracy threshold;

[0102] Step S433, if the accuracy rate is greater than the preset accuracy threshold, the reply content matches the test audio data.

[0103] It is understandable that existing vehicles increasingly use large language models to respond to users' questions and answers. Therefore, even for the same question, the content of each reply may not be exactly the same. In this case, it is impossible to test by setting fixed answers. Therefore, multiple large models are introduced in this embodiment. Specifically, different third-party open source models can be selected. The test audio and the reply content are input into each large model as a set of question and answer pairs. The large model determines whether the reply content correctly answers the question corresponding to the test audio, that is, whether the reply content is the answer corresponding to the test audio. If the reply content correctly answers the question corresponding to the test audio, the output indicates a correct matching result, otherwise the output indicates an incorrect matching result.

[0104] The accuracy rate indicates the comprehensive results of the various models on whether the response content is the answer corresponding to the test audio; specifically, the accuracy rates are:

[0105] p=m1x1+m2x2+…+m n x n

[0106] Among them, p is the accuracy, m is n is the matching result corresponding to the nth large model. If the result is correct, then m n is 1, the result is an error, then m n is 0; x n is the weight corresponding to the nth large model, and the sum of the weights corresponding to each large model is 1. The weights corresponding to different large models can be set based on actual needs. For example, the stronger the model capability, the greater the corresponding weight.

[0107] After the accuracy rate is determined, it can be compared with the preset accuracy threshold. If the accuracy rate is greater than the preset accuracy threshold, the reply content matches the test audio data. If the accuracy rate is less than or equal to the preset accuracy threshold, the reply content does not match the test audio data. The specific value of the preset accuracy threshold can be set based on actual needs.

[0108] In this embodiment, the answers to the open-ended large model can be tested to determine the correctness of the reply content.

[0109] Further, in a sixth embodiment of the vehicle voice testing method of the present invention proposed based on the first embodiment of the present invention, the step S40 includes the following steps:

[0110] Step S80, receiving a regression test instruction, and determining regression corpus data corresponding to the regression test instruction in the test corpus data;

[0111] Step S90, storing the regression corpus data in a regression corpus;

[0112] Step S100, if the regression test is triggered, the regression corpus data is used as the target corpus data for testing;

[0113] Step S110: after the test is completed, the regression corpus data is deleted from the regression corpus.

[0114] After the test is completed, the user may need to adjust the in-vehicle voice system and then perform regression testing. Therefore, the user can specify the test corpus data to be retained as regression corpus data, and perform regression testing through the regression corpus data when regression testing is required; at the same time, in order to maintain the cleanliness and efficiency of the database, the regression test data can be deleted after the regression test is completed. It is understandable that the regression test can be performed with reference to the aforementioned test steps and will not be repeated here.

[0115] Furthermore, after the test is completed, a test report can be generated, such as obtaining the data generated during the test and performing statistics in units of test corpus data, which can include information such as test version, module, number of executions, number of successful or failed recognitions, number of successful or failed replies, number of successful or failed executions, etc. The system generates a detailed report based on these data, and the data in the report can be filtered by different fields. The system will evaluate whether the execution result of the current corpus is successful and mark it according to the success rate standard set by the user. In addition, the test report system is connected to the company's internal BUG management system, and supports one-click billing operations for test corpora that failed to execute. Users can also filter different execution results and add them to the regression corpus with one click.

[0116] In other embodiments, the user can also perform random testing, such as Monkey mode; as can be seen from the foregoing description, before the test, the user needs to set specific parameters in the test requirement data. This type of test is generally targeted at specific scenarios, while in non-targeted test scenarios, random parameters can be used to perform the test to obtain more diverse test results; specifically, the user can select parameters that need to be clarified according to needs, while parameters not set by the user are randomly set by the system; in order to ensure the correct execution of the test, the user can be required to at least set the test duration or the number of basic corpus items, and the test will be stopped when the test duration or the number of basic corpus items is reached.

[0117] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0118] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0119] The present application also provides a vehicle-mounted voice testing device for implementing the above-mentioned vehicle-mounted voice testing method, and the vehicle-mounted voice testing device includes:

[0120] A first acquisition module is used to acquire test requirement data and determine target corpus data in the test requirement data;

[0121] A second acquisition module is used to acquire a trained test corpus generation model, and perform generalized corpus generation on the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the trained test corpus generation model is trained with generalized corpus data as samples, and the amount of the test corpus data is greater than the amount of the target corpus data;

[0122] A first generating module, configured to generate a test audio according to the test corpus data, and play the test audio so that the vehicle voice system can collect the test audio;

[0123] The third acquisition module is used for the feedback data generated by the in-vehicle voice system based on the test audio, and obtains the test result according to the test requirement data and the feedback data.

[0124] The vehicle-mounted voice testing device generates generalized test corpus data based on the target corpus data through the trained test corpus generation model, so that the test corpus data can cover more diverse scenarios, solving the problem of corpus limitations, and then performing audio testing based on the test corpus, so that the accuracy of the test can be improved, the depth and breadth of the test can be expanded, and the reliability of the vehicle-mounted voice system can be improved.

[0125] It should be noted that the first acquisition module in this embodiment can be used to execute step S10 in the embodiment of the present application, the second acquisition module in this embodiment can be used to execute step S20 in the embodiment of the present application, the first generation module in this embodiment can be used to execute step S30 in the embodiment of the present application, and the third acquisition module in this embodiment can be used to execute step S40 in the embodiment of the present application.

[0126] Furthermore, the device also includes:

[0127] The fourth acquisition module is used to obtain the trained corpus generalization model and basic corpus data;

[0128] A first generalization module, configured to generalize the basic corpus data by using the trained corpus generalization model to obtain the generalized corpus data corresponding to the basic corpus data;

[0129] The fifth acquisition module is used to acquire an initial test corpus generation model, and train the initial test corpus generation model using generalized corpus data as a sample to obtain the trained test corpus generation model.

[0130] Furthermore, the second acquisition module includes:

[0131] A first acquisition unit is used to acquire target corpus configuration data in the test requirement data, and use the target corpus configuration data as operation configuration data of the trained test corpus generation model, wherein the target corpus configuration data includes target corpus complexity and target corpus quantity;

[0132] The first generation unit is used to generate a generalized corpus for the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the corpus complexity of the test corpus data is the target corpus complexity, and the number of the test corpus data is the target corpus number.

[0133] Furthermore, the first generating module includes:

[0134] A second acquisition unit, used to acquire target language feature data in the test requirement data;

[0135] A first matching unit, used for matching a digital person corresponding to the target language feature data;

[0136] The first sending unit is used to send the test corpus data to the digital human, so that the digital human plays the test corpus data to generate the test audio.

[0137] Furthermore, the third acquisition module includes:

[0138] A first judging unit, used to judge whether the test audio data is of a question-and-answer type;

[0139] A first collection unit, configured to collect, if the test audio data is of a question-and-answer type, a reply content played by the in-vehicle voice system based on the test audio;

[0140] A second judgment unit, used to judge whether the reply content matches the test audio data;

[0141] The second matching unit is used to determine that the test result is passed if the reply content matches the test audio data.

[0142] Furthermore, the second determining unit includes:

[0143] A first acquisition subunit is used to acquire multiple large models, and input the test audio and the reply content into each of the large models respectively, so that the large model determines whether the reply content is the answer corresponding to the test audio, and obtains multiple matching results;

[0144] A first calculation subunit, used to calculate the accuracy rate corresponding to the matching result, and determine whether the accuracy rate is greater than a preset accuracy threshold;

[0145] The first execution subunit is used to match the reply content with the test audio data if the accuracy rate is greater than the preset accuracy threshold.

[0146] Furthermore, the device also includes:

[0147] A first receiving module, configured to receive a regression test instruction and determine regression corpus data corresponding to the regression test instruction in the test corpus data;

[0148] A first storage module, used for storing the regression corpus data into a regression corpus;

[0149] A first testing module, configured to test the regression corpus data as the target corpus data if a regression test is triggered;

[0150] The first deletion module is used to delete the regression corpus data from the regression corpus after the test is completed.

[0151] Reference Figure 8 In terms of hardware structure, the electronic device may include components such as a communication module 10, a memory 20, and a processor 30. In the electronic device, the processor 30 is connected to the memory 20 and the communication module 10 respectively, and a computer program is stored in the memory 20. The computer program is executed by the processor 30 at the same time, and the steps of the above method embodiment are implemented when the computer program is executed.

[0152] The communication module 10 can be connected to an external communication device through a network. The communication module 10 can receive requests from an external communication device, and can also send requests, instructions and information to the external communication device, which can be other electronic devices, servers or Internet of Things devices, such as televisions, etc.

[0153] The memory 20 can be used to store software programs and various data. The memory 20 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function (such as obtaining test requirement data), etc.; the data storage area can include a database, and the data storage area can store data or information created according to the use of the system, etc. In addition, the memory 20 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0154] The processor 30 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 20, and calling data stored in the memory 20, so as to monitor the electronic device as a whole. The processor 30 may include one or more processing units; optionally, the processor 30 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 30.

[0155] although Figure 8 Although not shown, the electronic device may further include a circuit control module, which is used to connect to a power source to ensure the normal operation of other components. Figure 8 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0156] The present invention also provides a computer-readable storage medium on which a computer program is stored. The computer-readable storage medium may be Figure 8 The memory 20 in the electronic device may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The computer-readable storage medium includes a number of instructions for enabling a terminal device with a processor (which may be a television, a car, a mobile phone, a computer, a server, a terminal, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0157] In the present invention, the terms "first", "second", "third", "fourth" and "fifth" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0158] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0159] Although the embodiments of the present invention have been shown and described above, the scope of protection of the present invention is not limited thereto. It is understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art can change, modify and replace the above embodiments within the scope of the present invention, and these changes, modifications and replacements should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A vehicle-mounted voice testing method, characterized in that: The vehicle-mounted voice testing method comprises: Acquire test requirement data, and determine target corpus data in the test requirement data; Acquire a trained test corpus generation model, and perform generalized corpus generation on the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the trained test corpus generation model is trained using generalized corpus data as samples, and the amount of the test corpus data is greater than the amount of the target corpus data; Generating a test audio according to the test corpus data, and playing the test audio so that the vehicle-mounted voice system can collect the test audio; Acquire feedback data generated by the in-vehicle voice system based on the test audio, and obtain a test result according to the test requirement data and the feedback data.

2. The vehicle-mounted voice testing method according to claim 1, characterized in that: Generating the test audio according to the test corpus data comprises: Acquire target language feature data in the test requirement data; Matching a digital person corresponding to the target language feature data; The test corpus data is sent to the digital human so that the digital human plays the test corpus data to generate the test audio.

3. The vehicle-mounted voice testing method according to claim 1, characterized in that: The step of obtaining the trained test corpus generation model includes: Obtain the trained corpus generalization model and basic corpus data; Generalizing the basic corpus data through the trained corpus generalization model to obtain the generalized corpus data corresponding to the basic corpus data; An initial test corpus generation model is obtained, and the initial test corpus generation model is trained using generalized corpus data as a sample to obtain the trained test corpus generation model.

4. The vehicle-mounted voice testing method according to claim 1, characterized in that: The test corpus generation model completed through the training performs generalized corpus generation on the target corpus data to obtain test corpus data corresponding to the target corpus data, including: Obtaining target corpus configuration data in the test requirement data, and using the target corpus configuration data as operation configuration data of the trained test corpus generation model, wherein the target corpus configuration data includes target corpus complexity and target corpus quantity; The generalized corpus generation model completed through the training performs generalized corpus generation based on the target corpus data to obtain test corpus data corresponding to the target corpus data, wherein the corpus complexity of the test corpus data is the target corpus complexity, and the number of the test corpus data is the number of the target corpus.

5. The vehicle-mounted voice testing method according to claim 1, characterized in that: The obtaining feedback data generated by the vehicle voice system based on the test audio, and obtaining a test result according to the test requirement data and the feedback data includes: Determining whether the test audio data is of a question-and-answer type; If the test audio data is of question and answer type, collecting the reply content played by the in-vehicle voice system based on the test audio; Determining whether the reply content matches the test audio data; If the reply content matches the test audio data, the test result is passed.

6. The vehicle-mounted voice testing method according to claim 5, characterized in that: The determining whether the reply content matches the test audio data comprises: Acquire multiple large models, and input the test audio and the reply content into each of the large models respectively, so that the large model determines whether the reply content is the answer corresponding to the test audio, and obtains multiple matching results; Calculate the correct rate corresponding to the matching result, and determine whether the correct rate is greater than a preset correct threshold; If the accuracy rate is greater than the preset accuracy threshold, the reply content matches the test audio data.

7. The vehicle-mounted voice testing method according to claim 1, characterized in that: The step of obtaining the test result according to the test requirement data and the feedback data includes: Receiving a regression test instruction, and determining regression corpus data corresponding to the regression test instruction in the test corpus data; Storing the regression corpus data in a regression corpus; If the regression test is triggered, the regression corpus data is used as the target corpus data for testing; After the test is completed, the regression corpus data is deleted from the regression corpus.

8. A vehicle-mounted voice testing device, characterized in that: The vehicle-mounted voice testing device comprises: A first acquisition module is used to acquire test requirement data and determine target corpus data in the test requirement data; A second acquisition module is used to acquire a trained test corpus generation model, and perform generalized corpus generation on the target corpus data through the trained test corpus generation model to obtain test corpus data corresponding to the target corpus data, wherein the trained test corpus generation model is trained with generalized corpus data as samples, and the amount of the test corpus data is greater than the amount of the target corpus data; A first generating module, configured to generate a test audio according to the test corpus data, and play the test audio so that the vehicle voice system can collect the test audio; The third acquisition module is used for the feedback data generated by the in-vehicle voice system based on the test audio, and obtains the test result according to the test requirement data and the feedback data.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps of the in-vehicle voice testing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the in-vehicle voice testing method according to any one of claims 1 to 7 are implemented.

Images