Intelligent cabin AI voice interaction test method, computer device, and storage medium

Abstract

The application discloses an intelligent cockpit AI voice interaction test method, a computer device and a storage medium. The test method is: based on the function analysis of the intelligent cockpit AI voice interaction system, an intelligent test database is established; the intelligent test database stores interaction test cases, the interaction test cases contain multiple generalized corpus texts, and the generalized corpus texts are assigned difficulty coefficient values; during the test, in response to an externally input test difficulty coefficient instruction, the intelligent test database intelligently calculates and filters the generalized corpus texts selected according to the test difficulty coefficient instruction based on the difficulty coefficient values of the preset generalized corpus texts, replaces the generalized corpus texts into voice output, and performs interaction test; an AI intelligent model is called to judge the interaction test result, and a score is given to the intelligent cockpit AI voice interaction system. The application realizes intelligent test of the intelligent cockpit AI voice interaction system, greatly improves test intelligence and test efficiency, and guarantees consistency and objectivity of a test evaluation system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle testing, in particular to an intelligent cockpit AI voice interaction test method, computer equipment and a storage medium. BACKGROUND

[0002] With the rapid development of intelligent car cockpit, human-vehicle interaction has entered a new stage, especially the intelligent cockpit AI voice interaction system has developed rapidly and gradually become an important carrier of cockpit user experience and the core position of product competition. Intelligent cockpit AI voice interaction is a necessary means to create high-level human-computer interaction. The problem that follows is how to test the effectiveness of the intelligent cockpit AI voice interaction system to evaluate the performance of the intelligent cockpit AI voice interaction system and provide more objective test evaluation for users, which has become a problem that must be solved.

[0003] At present, the test method of intelligent cockpit AI voice interaction mainly includes subjective test and objective test. The subjective test includes the most basic international MOS scoring method, which evaluates the product through the most intuitive user experience and terminal use effect, avoiding the difficulty of converting consumer language into engineering language. The objective test avoids the influence of environmental and human factors on experimental results through standardized and automated testing.

[0004] However, for the intelligent cockpit AI voice interaction system, how to test effectively to evaluate the performance of the intelligent cockpit AI voice interaction system, the current subjective and objective combined test technology cannot more comprehensively and more realistically reflect the product performance and user experience of the intelligent cockpit AI voice interaction system. Therefore, it is of great significance to develop a test method for the intelligent cockpit AI voice interaction system. SUMMARY

[0005] The purpose of the present application is to overcome the deficiencies and defects of the prior art and provide an intelligent cockpit AI voice interaction test method, computer equipment and a storage medium.

[0006] One object of the present application is to provide an intelligent cockpit AI voice interaction test method, which comprises the following steps:

[0007] Based on the analysis of the functions of the intelligent cockpit AI voice interaction system, an intelligent test database is established, which stores a plurality of different interactive test cases, each of which is used to test an interactive scenario of the intelligent cockpit AI voice interaction system, each interactive test case contains an interactive test scenario, each interactive test scenario includes a plurality of interactive tasks, each interactive task includes a plurality of interactive test tasks, each interactive test task contains a plurality of source corpus texts, and the plurality of source corpus texts are formed into a plurality of test corpus texts through generalization, and each generalization corpus text is assigned a difficulty coefficient value.

[0008] When testing the intelligent cockpit AI voice interaction system, the intelligent test database intelligently selects the generalization corpus text that meets the test difficulty coefficient instruction based on the difficulty coefficient value of the preset generalization corpus text, replaces it with a voice output, and performs interactive testing on the intelligent cockpit AI voice interaction system.

[0009] Collecting the interactive test results, the intelligent test database calls an AI intelligent model to judge the interactive test results, and gives a score to the intelligent cockpit AI voice interaction system according to the judgment result.

[0010] Wherein, when the intelligent test database calls an AI intelligent model to judge the interactive test results, the accuracy of the feedback results of the intelligent cockpit AI voice interaction system is judged according to the positive and negative correlation of the interactive results.

[0011] Wherein, according to the interactive situation, the intelligent cockpit AI voice interaction system is given a score, including the overall score of the intelligent cockpit AI voice interaction system and / or the scene score of each interactive scenario.

[0012] Wherein, the generalization corpus text is formed by a preset corpus generalization system, which processes the source corpus text through different regional colloquial expressions, extracts key information from colloquial expressions and converts it into keywords, and then forms it based on the keywords according to the preset rules; wherein the keywords are obtained offline or online by connecting the keyword library by the corpus generalization system, or obtained online from related websites / platforms through the network, and the keyword library includes place name information library, scenic spot name library, celebrity information library, cuisine name library, historical anecdotes library, music library, and encyclopedia knowledge base.

[0013] Wherein, when testing the intelligent cockpit AI voice interaction system, the intelligent test database intelligently selects the generalization corpus text from the test case according to the test difficulty coefficient instruction, and selects it according to the following method:

[0014] The number of source corpus texts under each interactive test task, the number of interactive tasks under each interactive test scene, and the number of interactive test scenes are consistent in each test.

[0015] The AI intelligent model is multiple, and different AI intelligent models are called by the intelligent test database to judge the interactive test result according to the attribute of the interactive test result.

[0016] For the same interactive test result, the intelligent test database calls at least two different AI intelligent models to judge the same interactive test result, cross-verification, and finally gives a positive and negative correlation degree judgment result.

[0017] The intelligent test database intelligently calculates and filters the generalized corpus texts that meet the test difficulty coefficient instruction based on the difficulty coefficient value of the preset generalized corpus text, and replaces the voice output to output the dialect voice or standard general voice corresponding to different regions to interactively test the intelligent cockpit AI voice interaction system.

[0018] The second object of the application provides a computer device, comprising a processor and a memory, the memory stores a computer program for calling and executing by the processor, and the computer program is executed by the processor to realize the intelligent cockpit AI voice interaction test method.

[0019] The third object of the application provides a storage medium storing a computer program, and the computer program is executed by the processor to realize the intelligent cockpit AI voice interaction test method.

[0020] The intelligent cockpit AI voice interaction test method of the application, through the intelligent test database, responds to the test difficulty coefficient instruction input from the outside, intelligently calculates and filters the generalized corpus texts that meet the test difficulty coefficient instruction based on the difficulty coefficient value of the preset generalized corpus text, replaces the voice output, and interacts with the intelligent cockpit AI voice interaction system, calls the AI intelligent model to judge the interactive test result, and gives the intelligent cockpit AI voice interaction system a score according to the interaction, which can realize the intelligent test of the intelligent cockpit AI voice interaction system, greatly improve the intelligence and efficiency of the test, and ensure the consistency and objectivity of the test evaluation system. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1is a flowchart of the intelligent cockpit AI voice interaction test method of the present application. DETAILED DESCRIPTION

[0022] The present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0023] Referring to Figure 1 In the exemplary embodiments of the present application, the intelligent cockpit AI voice interaction test method comprises the following steps:

[0024] S1. Based on the function of the intelligent cockpit AI voice interaction system, an intelligent test database is established, and the intelligent test database stores a plurality of different interaction test cases, each interaction test case is used to test an interaction scene of the intelligent cockpit AI voice interaction system, each interaction test case includes an interaction test scene, each interaction test scene includes a plurality of interaction tasks, each interaction task includes a plurality of interaction test tasks, each interaction test task includes a plurality of source corpus texts, the plurality of source corpus texts are formed into a plurality of test corpus texts through generalization, and each generalization corpus text is assigned a difficulty coefficient value;

[0025] S2. When testing the intelligent cockpit AI voice interaction system, in response to an externally input test difficulty coefficient instruction, the intelligent test database intelligently calculates and filters the generalization corpus texts that meet the test difficulty coefficient instruction based on the difficulty coefficient values of the preset generalization corpus texts, replaces them with voice output, and performs interaction testing on the intelligent cockpit AI voice interaction system;

[0026] S3. Collecting the interaction test results, the intelligent test database calls an AI intelligent model to judge the interaction test results, and gives a score to the intelligent cockpit AI voice interaction system according to the judgment result.

[0027] In the present application, the interactive test scene can be determined according to the needs of the test, the needs of the interaction and the research needs of the user, and formed by means such as scene classification and deduplication. For example, the interactive test scene is divided into six categories, including travel assistant, vehicle manager, entertainment recommendation, encyclopedia query, interesting chat companion and image-text generation. For example, the interactive tasks under the travel assistant test scene include destination query, route screening, passing point setting, road condition query, travel planning and business trip reservation. The interactive tasks under the destination query include specified location surrounding destination query, historical destination query, activity venue query, specific demand destination query, specific condition destination query, nearby destination query, destination query by rating, destination query by price, fuzzy semantic destination query, distance range limited destination query, combined condition destination query and multi-round destination recommendation. For example, the interactive tasks under the vehicle manager test scene include vehicle travel check, vehicle operation guidance, vehicle active service, important schedule reminder and telephone intelligent dialing. The interactive tasks under the vehicle travel check include door status check, tire pressure status check, endurance check and maintenance reminder. For example, the interactive tasks under the entertainment recommendation test scene include music recommendation, audio book recommendation, video recommendation, radio recommendation and game recommendation. The interactive tasks under the music recommendation include volume setting, opening / closing music, searching by singer, searching by year, searching by music chart, searching by music type, searching by music listener type, searching by user habit, combined condition music search, random music switching, context-based song switching and the like. For example, the interactive tasks under the encyclopedia query test scene include natural science, humanities and social sciences, engineering technology and life common sense. The interactive tasks under the natural science include biological knowledge, geographical knowledge and health knowledge. For example, the interactive tasks under the interesting chat companion test scene include child travel companion, adult emotion exchange, news hot question and answer and future weather query. The interactive tasks under the child travel companion include enlightenment training, story telling and emotion soothing. For example, the interactive tasks under the image-text generation test scene include custom picture generation and custom text generation. The interactive tasks under the custom picture generation include wallpaper generation, multi-round wallpaper generation and picture generation.

[0028] Wherein, each test task under each interactive test scene includes a plurality of source corpus texts for testing, and the source corpus texts form specific test corpus texts through generalization to form generalization corpus texts, each test corpus text corresponds to a different test task to complete the test of the test task. Specifically, the travel assistant test scene includes test tasks such as destination query, route screening, passing point setting, road condition query, travel planning, and business trip reservation, and the interactive test task of the specified location surrounding destination query under the destination query includes source corpus text that can be query

recommend [hotel] in [West Lake Scenic Area]

where is the concert of a certain star held next month

[0029] The above is only an example of description, which is not limited thereto, and the specific classification and the number of test corpus texts are not limited, which can be infinitely expanded by the intelligent test database based on the obtained keywords according to the corresponding rules.

[0030] In the embodiments of the present application, when testing the intelligent cockpit AI voice interactive system, the intelligent test database can output voice through the test robot connected thereto, including outputting different regional accents or dialects to test the intelligent cockpit AI voice interactive system, and the test robot collects the interactive feedback results of the intelligent cockpit AI voice interactive system test, together with the corresponding test corpus feedback back to the intelligent test database. The intelligent test database calls the AI intelligent model to judge the interactive test results.

[0031] In the example embodiments of the present application, when the intelligent test database calls an AI intelligent model to judge the interactive test results, the accuracy of the feedback results of the intelligent cockpit AI voice interaction system is judged according to the positive and negative correlation degrees of the main or key information contained in the interactive results. For example, when a ticket for a flight to Shanghai on Monday morning next week is queried, the main or key information such as the departure place and destination, flight information and departure time is extracted from the ticket information provided by the intelligent cockpit AI voice interaction system, an AI intelligent model is called to verify the positive and negative correlation degrees of the ticket information, and it is judged whether the correlation is positive or negative. If the correlation is positive, the result is correct, and if the correlation is negative, the result is incorrect. According to the preset rules of the relationship between the correlation degree and the score, the test results of the intelligent cockpit AI voice interaction system are scored.

[0032] In the example embodiments of the present application, the intelligent cockpit AI voice interaction system is scored according to the interaction situation, including the overall score of the intelligent cockpit AI voice interaction system and / or the scene score of each interactive test scene. The intelligent test database in the present application can realize the test evaluation of the intelligent cockpit AI voice interaction system based on the intelligent verification of the AI intelligent model according to the interactive response results, give a score according to the positive and negative correlation degrees of the evaluation results of the interactive results, and then score the test results of the overall intelligent cockpit AI voice interaction system according to the score, or score the test results of the intelligent cockpit AI voice interaction system for different scenes, so as to realize the objective scoring of the overall and partial performance of the intelligent cockpit AI voice interaction system, and facilitate the user to improve the intelligent cockpit AI voice interaction system or improve the performance.

[0033] In the embodiments of the present application, the intelligent test database can also be connected with a response information collection module configured for testing, and the response information of the intelligent cockpit AI voice interaction system is collected through the response information collection module, which can be a functional module of a test robot or a configured intelligent collection robot connected with the intelligent test database, including voice response information and screen output response information. The response information collection module includes an image collection module and a sound collection module, and the collected response information is used to judge the positive and negative correlation degrees of the response of the intelligent cockpit AI voice interaction system by the AI intelligent model, so as to realize the response evaluation of the intelligent cockpit AI voice interaction system.

[0034] In the exemplary embodiments of the present application, the generalized corpus text can be formed by a preset corpus generalization system through processing the collected source corpus text in different regional colloquial expressions, extracting key information from the colloquial expressions, and converting the key information into keywords, and then forming the keywords based on the preset rules. The corpus generalization system can be a separate system connected to the intelligent test database to form generalized corpus text from the source corpus text, and then input the formed generalized corpus text into the intelligent test database for storage, or the corpus generalization system can be a sub-unit of the intelligent test database.

[0035] In the process of corpus text generalization, the corpus generalization system analyzes the expression of the input source corpus text, obtains the key information therein, forms keywords based on the key information, and forms multiple different expressions based on the keywords, thereby forming multiple generalized corpus texts. For example, the source corpus text is "go to the [coffee shop] that I went to last time", and after the colloquial expression is formed, the key information (coffee shop) is obtained by analysis, then the keyword "coffee shop" is formed based on the key information, and finally multiple generalized corpus texts are formed, such as "I want to drink coffee, navigate to the coffee shop I went to last time", "I am a little tired, take me to the coffee shop I went to last time to rest, navigate to the coffee shop I went to last time, find the coffee shop I went to last time, and then navigate there". Thus, the corpus library of generalized corpus text is greatly enriched, and the intelligence of the test is improved.

[0036] In the exemplary embodiments of the present application, the keywords can also be obtained offline or online by the corpus generalization system connecting the keyword library, or obtained online from related websites / platforms through the network, and then the keywords are replaced based on the obtained keywords, and the generalized corpus text is formed based on the source corpus text and the generalization rules. Thus, the multiple generalized corpus texts can be automatically updated, the generalized corpus text library can be expanded, and the keyword library includes geographic name information library, scenic spot name library, celebrity information library, cuisine name library, historical anecdotes library, encyclopedia knowledge library, picture information library, music collection library, music melody library, etc. which can obtain the required keyword database or collection. Subsequently, the keyword library can be added or modified according to the use of the corpus library, without limitation.

[0037] In the exemplary embodiments of the present application, when testing the intelligent cockpit AI voice interaction system, the intelligent test database extracts the generalized corpus text that meets the test difficulty coefficient from the library based on the externally input test difficulty coefficient, and tests the intelligent cockpit AI voice interaction system through voice output.

[0038] In the example embodiment of the present application, when testing the intelligent cockpit AI voice interaction system, the preset scene difficulty coefficient is selected according to the test difficulty coefficient instruction when intelligently screening the generalized corpus text from the test case in the following manner:

[0039] During each test, the number of source corpus texts under each interactive test task, the number of interactive test tasks under each interactive test scene, and the number of interactive test scenes are consistent. Under the above screening prerequisite, the test difficulty coefficient is calculated according to the difficulty coefficient value of the screened generalized corpus text, so that the total difficulty coefficient of the selected generalized corpus text corresponds to the test difficulty coefficient.

[0040] The number of corpus interaction rounds under each interactive test task can be set according to the needs of each interactive test task, and is at least one round, which can be two rounds or three rounds, or multiple rounds, and is set according to the test needs of different interactive test scenes. During the setting, one interactive round can correspond to one source corpus text, and two interactive rounds can correspond to two source corpus texts. For example, in the case of two rounds, the source corpus text under the volume setting interactive test task of music recommendation is "play a song" + "the music is a bit loud", and in the case of one round, the source corpus text under the interactive test task of searching for music by age is "search for [classic songs] in [the 1980s]".

[0041] In the example embodiment of the present application, the AI intelligent model is multiple, and the intelligent test database calls different AI intelligent models to judge the interactive test result according to the attribute of the interactive test result. The matching AI intelligent model is used to judge the interactive test result, that is, at least one matching AI intelligent model is called to judge whether an interactive test result is successful or not. When calling, the matching AI intelligent model can be called according to the applicable scene or range of the AI intelligent model, or its use field or professional field, and the relevance of the interactive test to ensure the accuracy of the judgment of the interactive test result.

[0042] In the example embodiment of the present application, the AI intelligent model can be embedded in the intelligent test database. After the test information collection device feeds back the test result to the intelligent test database, the intelligent test database calls the AI intelligent model to judge the interactive test result, and the matching AI intelligent model is used to judge the interactive test result. The AI intelligent model can also be arranged independently of the intelligent test database and connected through a communication system to realize calling and judgment.

[0043] For the same interaction test result, the intelligent test database calls at least two different matched AI intelligent models to judge the same interaction test result, cross- verifies, and finally gives a positive or negative correlation degree judgment result. When the two different AI intelligent models give a positive correlation evaluation, the positive correlation evaluation of the interaction response result is given. If the two different AI intelligent models give opposite correlation evaluations, a third AI intelligent model is called to judge the same interaction test result, and the final judgment result is given according to the judgment result of the third AI intelligent model. If the judgment result of the third AI intelligent model is positive correlation, it is finally considered as positive correlation. If it is negative correlation, it is finally determined as negative correlation. Through the judgment and interaction verification of multiple AI intelligent models, the accuracy or correlation of the judgment is improved. It can also be more different AI intelligent models under an odd number, such as 5, 7, etc. The final result is obtained according to the number of positive and negative correlations, and the more the number is, the final result is.

[0044] In the example embodiment of the application, the intelligent test database intelligently calculates and filters the generalized corpus text that meets the test difficulty coefficient instruction based on the preset difficulty coefficient value of the generalized corpus text, and replaces it with voice output. Through the connection of the voice conversion system, the generalized corpus text is converted into different accents or dialects through the voice conversion system, so as to output the dialect voice or standard Mandarin voice corresponding to different regions to interact with the intelligent cockpit AI voice interaction system. By outputting the voice as a dialect or a general language or other voice, the recognition ability and response ability of the intelligent cockpit AI voice interaction system to different voices or accents can be tested.

[0045] The example embodiment of the application also provides a computer device including a processor and a memory, the memory storing a computer program for execution by the processor, and the computer program is executed by the processor to implement the intelligent cockpit AI voice interaction test method.

[0046] The example embodiment of the application also provides a storage medium storing a computer program, the computer program being executed by the processor to implement the intelligent cockpit AI voice interaction test method. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0047] The intelligent cockpit AI voice interaction test method of the example embodiment of the present application can realize intelligent test of the intelligent cockpit AI voice interaction system, greatly improve the intelligence and test efficiency of the test, and ensure the consistency and objectivity of the test evaluation system.

[0048] The above shows and describes the basic principles and main features of the present application and the advantages of the present application, and it is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be realized in other specific forms without departing from the spirit or essential characteristics of the present application.

[0049] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application.

[0050] In addition, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description manner of the specification is only for the sake of clarity, those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be properly combined to form other embodiments that those skilled in the art can understand.

Claims

1. A method for testing AI voice interaction of an intelligent cabin, characterized in that, Comprising the following steps: Based on the function of the intelligent cockpit AI voice interaction system, an intelligent test database is established; the intelligent test database stores a plurality of different interactive test cases, each interactive test case is used to test an interactive scenario of the intelligent cockpit AI voice interaction system, each interactive test case includes an interactive test scenario, and the interactive test scenario at least includes six categories of travel assistants, vehicle managers, entertainment recommendations, encyclopedia queries, interesting conversations, and image-text generation; each interactive test scenario includes a plurality of interactive tasks, each interactive task includes a plurality of interactive test tasks, each interactive test task includes a plurality of source corpus texts, the plurality of source corpus texts are formed into a plurality of test generalization corpus texts through generalization, and each generalization corpus text is assigned a difficulty coefficient value; When testing the intelligent cockpit AI voice interaction system, the intelligent test database intelligently selects the generalization corpus text that meets the test difficulty coefficient instruction based on the difficulty coefficient value of the preset generalization corpus text, replaces it with a voice output, and performs interactive testing on the intelligent cockpit AI voice interaction system; Collecting the interactive test results, the intelligent test database calls an AI intelligent model to judge the interactive test results, and gives a score to the intelligent cockpit AI voice interaction system according to the judgment result; The AI intelligent model is a plurality of AI intelligent models, and the intelligent test database calls different AI intelligent models to judge the interactive test results according to the attributes of the interactive test results, and judges the interactive test results through the matched AI intelligent models.

2. The intelligent cabin AI voice interaction test method according to claim 1, wherein, When the intelligent test database calls the AI intelligent model to judge the interactive test results, whether the feedback result of the intelligent cockpit AI voice interaction system is accurate is judged according to the positive and negative correlation of the interactive results.

3. The intelligent cabin AI voice interaction test method according to claim 1, characterized in that, According to the interaction, the intelligent cockpit AI voice interaction system is given a score, including the overall score of the intelligent cockpit AI voice interaction system and / or the scene score of each interactive scenario.

4. The intelligent cabin AI voice interaction test method of claim 1, wherein, The generalization corpus text is formed by a preset corpus generalization system, which processes the source corpus text through different regional colloquial expressions, extracts key information from the colloquial expressions, converts the key information into keywords, and forms the generalization corpus text based on the keywords according to the preset rules; wherein the keywords are obtained offline or online by connecting the keyword library by the corpus generalization system, or obtained online from related websites / platforms through the network, and the keyword library includes place name information library, scenic spot name library, celebrity information library, cuisine name library, historical anecdotes library, music library, and encyclopedia knowledge base.

5. The intelligent cabin AI voice interaction test method according to claim 1, wherein, When testing the intelligent cockpit AI voice interaction system, the intelligent test database intelligently selects the generalization corpus text from the test cases according to the test difficulty coefficient instruction, and selects the generalization corpus text according to the following method: During each test, all interactive test scenarios, all interactive tasks under each interactive test scenario, all interactive test tasks under each interactive task, and the number of source corpus texts under each interactive test task remain consistent, and the number of corpus interaction rounds is consistent; According to the difficulty coefficient value calculation of the generalization corpus text, the total difficulty coefficient calculated by the selected generalization corpus text is finally consistent with the test difficulty coefficient.

6. The intelligent cabin AI voice interaction test method of claim 1, wherein, For the same interactive test result, the intelligent test database calls at least two different AI intelligent models to judge the same interactive test result, cross-verification, and finally gives a positive and negative correlation degree judgment result.

7. The intelligent cabin AI voice interaction test method according to claim 1, wherein, Based on the difficulty coefficient value of the preset generalization corpus text, the intelligent test database intelligently calculates and selects the generalization corpus text that meets the test difficulty coefficient instruction, and when the generalization corpus text is replaced as voice output, outputs the dialect voice or standard general voice corresponding to different regions to interact with the intelligent cockpit AI voice interaction system.

8. Computer device, characterized in that The computer program is stored in the memory and is called and executed by the processor, and the computer program is executed by the processor to implement the intelligent cockpit AI voice interaction test method in any one of claims 1 to 7.

9. Storage medium, characterized in that The computer program is stored in the memory and is called and executed by the processor, and the computer program is executed by the processor to implement the intelligent cockpit AI voice interaction test method in any one of claims 1 to 7.

Images