Test set construction method and device of intelligent cockpit voice model and electronic equipment

Abstract

The invention provides a test set construction method and device for an intelligent cockpit voice model and electronic equipment. The method comprises the following steps: acquiring user behavior data of a target vehicle; performing mining processing based on the user behavior data to obtain an initial test data set; performing tuning processing on the initial test data set to obtain a tuning test data set; performing verification processing on the tuning test data set to obtain a quality verification result; and determining a target test data set by using the quality verification result. According to the invention, the technical problems of unbalanced function point ratio and poor data quality existing in the test set construction method of the intelligent cockpit voice model provided in the related technology are solved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of smart cockpits, and in particular to a test set construction method, device and electronic equipment for a smart cockpit speech model. Background Art

[0002] With the continuous advancement of smart cockpit technology, voice interaction models have become a key factor in optimizing user experience. The construction of the voice model test set directly affects the efficiency and convenience of communication between users and vehicles. Therefore, creating a high-quality voice model test set is crucial to improving the interactive experience of the smart cockpit. In related technologies, the construction of test sets for smart cockpit voice models usually relies on two methods: one is to manually write test cases based on product functions; the other is to directly use the interaction logs of online users as test sets. However, the two methods in related technologies have problems with unbalanced functional point proportions and poor data quality, which reduces the accuracy of voice interaction.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present disclosure provide a method, device and electronic device for constructing a test set for an intelligent cockpit voice model, so as to at least solve the technical problems of unbalanced functional point proportion and poor data quality existing in the method for constructing a test set for an intelligent cockpit voice model provided in the related art.

[0005] According to one aspect of an embodiment of the present disclosure, a test set construction method for a smart cockpit voice model is provided, including: obtaining user behavior data of a target vehicle, wherein the user behavior data is used to represent voice interaction data of the target vehicle within a preset time period; performing mining processing based on the user behavior data to obtain an initial test data set; performing tuning processing on the initial test data set to obtain a tuned test data set, wherein the tuned test data is used to represent a result of a proportion adjustment of function points corresponding to the initial test data set; performing verification processing on the tuned test data set to obtain a quality verification result, wherein the quality verification result is used to determine whether the tuned test data set meets a preset acceptance standard; and determining a target test data set using the quality verification result, wherein the target test data set is used to perform a functional coverage test on the smart cockpit voice model.

[0006] Optionally, mining is performed based on user behavior data to obtain an initial test data set, including: filtering the user behavior data using domain intent association rules to obtain a first filtering result, wherein the first filtering result is used to represent valid intent data in the user behavior data; normalizing the first filtering result to obtain a normalized result, wherein the normalized result is used to represent a sentence normalization result corresponding to the first filtering result; aggregating the normalized result according to the user intent dimension to obtain a function point distribution statistical result; and determining an initial test data set based on the function point distribution statistical result.

[0007] Optionally, the first screening result is normalized to obtain the normalized result, including: performing data identification on the first screening result to obtain key data information, wherein the key data information includes key sentence information and keyword information corresponding to the first screening result; and performing sentence normalization processing using the key data information to obtain the normalized result.

[0008] Optionally, the normalization processing results are aggregated according to the user intention dimension to obtain the function point distribution statistics including: determining at least one target user intention based on the normalization processing results; obtaining a first access parameter corresponding to the at least one target user intention, wherein the first access parameter is used to represent the number of independent visits corresponding to the at least one target user intention; determining a second access parameter corresponding to the voice function point based on the first access parameter, wherein the voice function point is associated with the at least one target user intention, and the second access parameter is used to represent the number of independent visits corresponding to the voice function point; and determining the function point distribution statistics according to the second access parameter.

[0009] Optionally, mining the user behavior data to obtain an initial test data set also includes: filtering the user behavior data using domain intent association rules to obtain a second filtering result, wherein the second filtering result is used to represent invalid intent data in the user behavior data; determining a first keyword based on the first filtering result, and determining a second keyword based on the second filtering result; determining a supplementary test data set based on the first keyword and the second keyword, wherein the supplementary test data set is used to supplement the tuning test data set.

[0010] Optionally, determining the first keyword based on the first screening result includes: obtaining word frequencies and inverse document frequencies corresponding to multiple candidate keywords in the first screening result; and determining the first keyword based on the word frequencies and inverse document frequencies.

[0011] Optionally, the initial test data set is tuned to obtain the tuned test data set, including: obtaining a first function point and a second function point based on the initial test data set, wherein the number of independent accesses corresponding to the first function point is greater than a preset threshold, and the number of independent accesses corresponding to the second function point is less than the preset threshold; performing peak shaving processing on a first ratio corresponding to the first function point to obtain a first adjustment result; performing supplementary processing on a second ratio corresponding to the second function point to obtain a second adjustment result; and determining the tuned test data set using the first adjustment result and the second adjustment result.

[0012] Optionally, performing verification processing on the tuned test data set to obtain a quality verification result includes: performing a first verification processing on the tuned test data set to obtain a first verification result, wherein the first verification result is used to represent a label verification result corresponding to the tuned test data set; performing a second verification processing on the tuned test data set to obtain a second verification result, wherein the second verification result is used to represent the degree of distribution proximity between the tuned test data set and the user behavior data; and determining the quality verification result based on the first verification result and the second verification result.

[0013] According to one embodiment of the present disclosure, a test set construction device for an intelligent cockpit voice model is also provided, including: an acquisition module, used to acquire user behavior data of a target vehicle, wherein the user behavior data is used to represent voice interaction data of the target vehicle within a preset time period; a mining module, used to perform mining processing based on the user behavior data to obtain an initial test data set; a tuning module, used to perform tuning processing on the initial test data set to obtain a tuned test data set, wherein the tuned test data is used to represent a function point ratio adjustment result corresponding to the initial test data set; a verification module, used to perform verification processing on the tuned test data set to obtain a quality verification result, wherein the quality verification result is used to determine whether the tuned test data set meets a preset acceptance standard; a determination module, used to determine a target test data set using the quality verification result, wherein the target test data set is used to perform functional coverage testing on the intelligent cockpit voice model.

[0014] Optionally, the mining module is also used to: use domain intent association rules to filter user behavior data to obtain a first filtering result, wherein the first filtering result is used to represent valid intent data in the user behavior data; normalize the first filtering result to obtain a normalized result, wherein the normalized result is used to represent a sentence normalization result corresponding to the first filtering result; aggregate the normalized result according to the user intent dimension to obtain a function point distribution statistical result; and determine an initial test data set based on the function point distribution statistical result.

[0015] Optionally, the mining module is also used to: perform data identification on the first screening result to obtain key data information, wherein the key data information includes key sentence information and keyword information corresponding to the first screening result; perform sentence normalization processing using the key data information to obtain a normalized processing result.

[0016] Optionally, the mining module is also used to: determine at least one target user intention based on the normalization processing result; obtain a first access parameter corresponding to at least one target user intention, wherein the first access parameter is used to represent the number of independent accesses corresponding to at least one target user intention; determine a second access parameter corresponding to a voice function point based on the first access parameter, wherein the voice function point is associated with at least one target user intention, and the second access parameter is used to represent the number of independent accesses corresponding to the voice function point; determine the function point distribution statistics according to the second access parameter.

[0017] Optionally, the mining module is also used to: use domain intent association rules to filter and process user behavior data to obtain a second filtering result, wherein the second filtering result is used to represent invalid intent data in the user behavior data; determine a first keyword based on the first filtering result, and determine a second keyword based on the second filtering result; determine a supplementary test data set based on the first keyword and the second keyword, wherein the supplementary test data set is used to supplement the tuning test data set.

[0018] Optionally, the mining module is further used to: obtain word frequencies and inverse document frequencies corresponding to multiple candidate keywords in the first screening result; and determine the first keyword based on the word frequencies and inverse document frequencies.

[0019] Optionally, the tuning module is also used to: obtain a first function point and a second function point based on an initial test data set, wherein the number of independent accesses corresponding to the first function point is greater than a preset threshold, and the number of independent accesses corresponding to the second function point is less than the preset threshold; perform peak shaving processing on a first ratio corresponding to the first function point to obtain a first adjustment result; perform supplementary processing on a second ratio corresponding to the second function point to obtain a second adjustment result; and determine a tuning test data set using the first adjustment result and the second adjustment result.

[0020] Optionally, the verification module is also used to: perform a first verification process on the tuned test data set to obtain a first verification result, wherein the first verification result is used to represent the annotation verification result corresponding to the tuned test data set; perform a second verification process on the tuned test data set to obtain a second verification result, wherein the second verification result is used to represent the degree of distribution closeness between the tuned test data set and the user behavior data; and determine the quality verification result based on the first verification result and the second verification result.

[0021] According to one embodiment of the present disclosure, there is also provided an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute instructions to implement the test set construction method for the smart cockpit speech model in the embodiment of the present disclosure.

[0022] According to one embodiment of the present disclosure, a computer-readable storage medium is also provided, which includes a stored executable program, wherein when the executable program is running, the device where the storage medium is located is controlled to execute the test set construction method for the smart cockpit speech model in the embodiment of the present disclosure.

[0023] According to one embodiment of the present disclosure, a computer program product is also provided, including a computer program, which, when executed by a processor, implements the test set construction method for the smart cockpit speech model in the embodiment of the present disclosure.

[0024] In the disclosed embodiment, the user behavior data of the target vehicle is obtained, and mining is performed based on the user behavior data to obtain an initial test data set, and then the initial test data set is tuned to obtain an optimized test data set, and the optimized test data set is verified to obtain a quality verification result, and finally the target test data set is determined using the quality verification result to ensure that the generated test set is user representative and high quality. The function coverage test of the smart cockpit voice model is performed using the target test data set, achieving the purpose of optimizing the smart cockpit voice model, thereby achieving the technical effect of balancing the functional proportion of the test set and improving the data quality, and thus solving the technical problems of unbalanced functional point proportion and poor data quality in the test set construction method of the smart cockpit voice model provided in the related art. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0026] Figure 1 is a flowchart of a test set construction method for an intelligent cockpit speech model according to one embodiment of the present disclosure;

[0027] Figure 2 is a schematic diagram of a domain intention association rule according to one embodiment of the present disclosure;

[0028] Figure 3 is a schematic diagram of a normalization process according to one embodiment of the present disclosure;

[0029] Figure 4 is a schematic diagram of a construction process of a supplementary test set according to one embodiment of the present disclosure;

[0030] Figure 5 is a schematic diagram of a test set construction method for an intelligent cockpit speech model according to one embodiment of the present disclosure;

[0031] Figure 6 It is a structural block diagram of a test set construction device for an intelligent cockpit speech model according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0034] In the related art, the test set construction of the smart cockpit voice model usually relies on two methods: one is to manually write test cases based on product functions; the other is to directly use the interaction logs of online users as test sets. However, the two methods in the related art have problems such as unbalanced functional point ratios and poor data quality, which reduces the accuracy of voice interaction.

[0035] Specifically, although the test set manually written based on product functions can ensure that the test set covers all functional points, there are the following problems: the constructed data set is often based on the preset understanding and standardized terms of the function, which may be inconsistent with the user's natural language expression habits, resulting in the test set not matching the user's actual usage scenario, and the manually written method is difficult to cover all possible user expressions, which is insufficient to reflect the performance of the model in a complex and changeable real environment, which may cause the tester to unconsciously favor certain functional points, resulting in the distribution of functions in the test set not matching the actual user usage. Simply using online log data as the test set for the voice model may also result in too many test samples for high-frequency functional points and insufficient samples for low-frequency functional points, thus affecting the comprehensiveness and balance of the test. In addition, the construction of existing test sets lacks strict control over data quality and may contain a large amount of invalid or low-quality data, further affecting the test results.

[0036] According to an embodiment of the present disclosure, a method embodiment of a method for constructing a test set for an intelligent cockpit speech model is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0037] Figure 1 is a flowchart of a test set construction method for an intelligent cockpit speech model according to one embodiment of the present disclosure, such as Figure 1 As shown, the method comprises the following steps:

[0038] S11, obtaining user behavior data of the target vehicle, wherein the user behavior data is used to represent voice interaction data of the target vehicle within a preset time period;

[0039] S12, performing mining processing based on user behavior data to obtain an initial test data set;

[0040] S13, performing tuning processing on the initial test data set to obtain a tuned test data set, wherein the tuned test data is used to represent a function point ratio adjustment result corresponding to the initial test data set;

[0041] S14, verifying the tuning test data set to obtain a quality verification result, wherein the quality verification result is used to determine whether the tuning test data set meets a preset acceptance standard;

[0042] S15, determining a target test data set using the quality verification result, wherein the target test data set is used to perform functional coverage testing on the smart cockpit speech model.

[0043] The target vehicles mentioned above refer to a specific set of vehicles used for data collection. In the process of selecting target vehicles, relevant laws and regulations on data compliance and user privacy protection must be strictly observed. By selecting target vehicles that meet certain conditions, it can be ensured that the collected data can not only effectively support the testing and improvement of the smart cockpit voice model, but also protect the user's personal information security and avoid user privacy leakage.

[0044] For example, we select the delivered models within the past year, and randomly select 25% based on the unique vehicle identification ID. The selected vehicles are delivered within three months to ensure that the collected voice interaction data can reflect the user's usage habits and diversity of statements in the early stage of new car delivery. The user interaction data in this period has not been affected by model training, so it can provide a more realistic and uncontaminated user behavior sample. At the same time, the data in this period covers all seasons, ensuring that some functional points that are only used in specific seasons are covered, such as air conditioning cooling and air conditioning heating.

[0045] The above user behavior data refers to the data collected by the smart cockpit system during the actual use of the user and the system interaction. This data includes information such as the user's voice commands, operating habits, and functional points used, which can reflect the user's real needs and expressions in different scenarios. In the context of the smart cockpit, user behavior data mainly comes from the vehicle's online log, which records the user's voice input and system response during driving.

[0046] For example, under the premise of ensuring that the user has explicitly authorized and complies with the "Experience Improvement Plan" agreement, the interaction data sent by the user is extracted from the cockpit interaction log of the authorized vehicle. Specifically, the interaction data between the user and the smart cockpit system is collected through the vehicle's built-in monitoring points, namely "buried points", and the obtained interaction data is used as the original data source for building the test set.

[0047] The above-mentioned mining process is the process of conducting data mining based on the actual usage ratio of the function points, including but not limited to data preprocessing, feature extraction, data analysis and data screening. By extracting information and sample data related to specific function points from user behavior data, the original and disorganized user logs are converted into structured and meaningful test data, ensuring that the extracted data samples can accurately reflect the user's real intentions and usage scenarios, and forming a test set that is both in line with the user's actual usage ratio and highly representative.

[0048] The above initial test data set is a data set initially generated after the mining process is completed for testing the smart cockpit voice model. The initial test data set contains samples directly extracted from user behavior data, which may be unbalanced or defective in the coverage of functional points, data quality and distribution, and needs further processing to improve its representativeness and effectiveness.

[0049] The above-mentioned tuning process refers to the process of further optimizing and adjusting the initial test data set obtained after the mining process, with the aim of ensuring that the test set can more accurately and comprehensively reflect actual user usage while maintaining the balance and effectiveness of the data set.

[0050] The above function point ratio adjustment result refers to the data set obtained by adjusting the ratio of different function points in the data set through certain algorithms and strategies based on the initial test data set. The purpose of function point ratio adjustment is to balance the distribution of function points in the data set, ensure that high-frequency function points do not excessively dominate the test results, and also ensure that low-frequency but important function points are adequately covered by the test.

[0051] The above-mentioned verification process refers to the process of quality inspection and performance evaluation of the adjusted test data set. It usually includes data re-labeling, statistical analysis, model testing and other links to ensure that the adjusted data set can truly reflect user usage and ensure that the data quality meets the preset standards. The purpose of verification processing is to further improve the accuracy and reliability of the test data set.

[0052] The above-mentioned preset acceptance indicators refer to a series of standards set during the data quality acceptance process to determine whether the test data set meets the conditions for final use. The preset acceptance indicators include, but are not limited to, key performance indicators such as data annotation accuracy, function point coverage, and model prediction accuracy. By setting and strictly enforcing these preset acceptance indicators, it can be ensured that the generated test set is of high quality and representativeness, thereby effectively supporting the testing and optimization process of the smart cockpit voice model and improving the user experience.

[0053] The above-mentioned target test data set refers to the data set used to formally test the smart cockpit voice model after a series of processes such as mining, function point ratio adjustment, verification, and data quality acceptance. The construction of the target test data set aims to ensure that the user statements it contains can fully cover the various functions of the smart cockpit voice model and are highly consistent with the actual user usage scenarios, thereby providing a solid foundation for the testing and optimization of the model, thereby improving the performance of the voice model and user experience.

[0054] Through the above steps S11 to S15, by obtaining the user behavior data of the target vehicle and performing mining processing based on the user behavior data, an initial test data set is obtained, and then the initial test data set is tuned to obtain a tuned test data set, and the tuned test data set is verified to obtain a quality verification result, and finally the target test data set is determined using the quality verification result to ensure that the generated test set is user representative and high quality. The function coverage test of the smart cockpit voice model is performed using the target test data set, achieving the purpose of optimizing the smart cockpit voice model, thereby achieving the technical effect of balancing the functional proportion of the test set and improving the data quality, thereby solving the technical problems of unbalanced functional point proportion and poor data quality in the test set construction method of the smart cockpit voice model provided in the related art.

[0055] The following is a further introduction to the test set construction method of the smart cockpit speech model in the embodiment of the present disclosure.

[0056] Optionally, in step S12, mining is performed based on the user behavior data to obtain an initial test data set including:

[0057] Step S121, using the domain intention association rule to filter the user behavior data to obtain a first filtering result, wherein the first filtering result is used to represent valid intention data in the user behavior data;

[0058] Step S122, normalizing the first screening result to obtain a normalized processing result, wherein the normalized processing result is used to represent a sentence normalization result corresponding to the first screening result;

[0059] Step S123, performing aggregation processing on the normalization processing results according to the user intention dimension to obtain the function point distribution statistical results;

[0060] Step S124: determining an initial test data set based on the function point distribution statistics result.

[0061] The domain intention association rules mentioned above refer to the logical criteria for associating the user's instructions or inquiry intentions with specific functional areas in the smart cockpit voice recognition system. They are the key to enabling the smart cockpit voice interaction system to efficiently and accurately respond to user instructions. The domain intention association rules ensure that the smart cockpit voice model can not only understand the user's intentions, but also accurately match them with the functional areas in the cockpit, thereby providing a more personalized and efficient user experience.

[0062] For example, when a user issues a voice command of "navigate to XX", the command is associated with the "navigation" domain, and specific intentions such as "keyword search" and "route switching" are parsed. By establishing association rules between domains and intentions, the system can not only identify the surface semantics of user commands, but also deeply understand the deep meaning behind user intentions, which helps the system provide more accurate and personalized service responses and enhance user experience.

[0063] Figure 2 is a schematic diagram of a domain intention association rule according to one embodiment of the present disclosure, such as Figure 2 As shown, the domain intention association rules include the domain classification and intention classification corresponding to the global function of voice, among which the domain classification corresponding to the global function of voice includes but is not limited to navigation, music, air conditioning, etc., the intention corresponding to the navigation field includes but is not limited to keyword search, route switching, etc.; the intention corresponding to the music field includes but is not limited to reading search, music search, etc.; the intention corresponding to the air conditioning field includes but is not limited to temperature adjustment, wind direction adjustment, etc.

[0064] The above-mentioned screening process refers to the process of extracting data that meets the requirements from a large amount of raw data according to specific standards or conditions during the data preprocessing stage. When building a test set for the smart cockpit voice model, screening is usually used to exclude invalid data and retain user statements that are related to functional points or can effectively test model performance. For example, remove conversations that are not related to the current test purpose, or filter out instructions that are too short and cannot express the full intent.

[0065] The above-mentioned valid intent data refers to user statements that can accurately reflect user needs or intentions and are directly related to the functional points of the smart cockpit voice model. In the construction of the test set, the valid intent data can provide real feedback on the model performance and help evaluate and optimize the accuracy and efficiency of the model in processing user natural language input.

[0066] Exemplarily, taking the voice recognition model for optimizing the air-conditioning control function as an example, data screening and processing are performed from the collected user voice logs according to specific conditions. First, through domain and intent recognition, all user statements related to air-conditioning control are extracted, such as "adjust the temperature to 22 degrees", "turn on the air conditioner", etc. Next, exclude those conversations that are not related to the air-conditioning control function, such as instructions for asking navigation or playing music. At the same time, filter out instructions that are too short or ambiguous, such as "warmer" and "cooler". Through screening and processing, it is ensured that the data in the test set are all valid instructions directly related to the air-conditioning control function, which not only ensures the quality of the test set, but also improves the pertinence and efficiency of subsequent model testing and optimization.

[0067] The above normalization is a data preprocessing method used to standardize the data in the data set to ensure that the data is compared and analyzed on the same scale. In the test set construction of the smart cockpit voice model, normalization is usually applied to process user statements, by converting instructions with different expressions but the same meaning into a unified format to reduce data redundancy and improve the efficiency and consistency of the data set.

[0068] Based on the above optional embodiments, the user behavior data is filtered and processed using domain intention association rules to obtain a first filtering result, and the first filtering result is normalized to obtain a normalized result, and then the normalized result is aggregated according to the user intention dimension to obtain a function point distribution statistical result, and finally, the initial test data set is determined based on the function point distribution statistical result, which can ensure that the test data set accurately reflects the user behavior characteristics of the smart cockpit voice model in the actual usage scenario, thereby improving the representativeness and effectiveness of the test data set.

[0069] Optionally, in step S122, the first screening result is normalized, and the normalized result obtained includes:

[0070] Step S1221, performing data recognition on the first screening result to obtain key data information, wherein the key data information includes key sentence information and keyword information corresponding to the first screening result;

[0071] Step S1222, use the key data information to perform sentence normalization processing to obtain a normalization processing result.

[0072] The above-mentioned data recognition refers to extracting and parsing information that is crucial for subsequent processing from the first screening results, including key sentence information and keyword information. Data recognition methods usually involve natural language processing (NLP) technology, such as part-of-speech tagging, named entity recognition, dependency syntax analysis, etc., to ensure that the core structure and key words of user instructions can be accurately identified.

[0073] For example, taking the original user statement "Navigate to XX University" as an example, data recognition is performed on the statement, and its key sentence pattern can be obtained as "Navigate to [keyword]", and the keyword is "XX University".

[0074] The above sentence normalization refers to converting different user expressions into a standard or canonical expression format to reduce the complexity of model training and testing caused by differences in expression. By converting different sentences with similar intents into the same sentence template while retaining their keyword information, it ensures that even if users use different expressions, the model can still recognize their intent and respond correctly, which helps improve the consistency of the data set and the generalization ability of the model.

[0075] For example, take the original user statements "Navigate to XX University" and "Navigate to XX Hotel" as examples. Although the specific locations are different, the two user statements both express the user's intention to use the navigation function to go to a specific location. In the normalization process, such user statements can be converted into a sentence template of "Navigate to [keyword]".

[0076] Figure 3 is a schematic diagram of a normalization process according to one embodiment of the present disclosure, such as Figure 3 As shown in the figure, in the process of normalizing user statements, it is first necessary to clean the original user statements, extract the key sentences in the original user statements, identify their keywords, and remove irrelevant information; secondly, perform sentence normalization based on the data recognition results, and output the sentence normalization results.

[0077] For example, taking the original user statements of "Navigate to XX University", "Navigate to XX Hotel", "I want to go to XX East Station", and "I want to go to XX Airport" as examples, data recognition and sentence normalization are performed on the above original user statements, and the sentence normalization results of "Navigate to [keyword]", "I want to go to [keyword]" are output to improve the efficiency of data processing and the effect of model training.

[0078] Based on the above optional embodiments, by performing data identification on the first screening result to obtain key data information, and then using the key data information to perform sentence normalization processing to obtain a normalized processing result, redundant data in the test set can be reduced, and repeated training of the model on the same intent can be avoided, thereby saving computing resources and time and improving overall testing efficiency.

[0079] Optionally, in step S123, the normalization processing results are aggregated according to the user intention dimension to obtain the function point distribution statistics including:

[0080] Step S1231, determining at least one target user intention based on the normalization processing result;

[0081] Step S1232, obtaining a first access parameter corresponding to at least one target user intention, wherein the first access parameter is used to represent the number of independent accesses corresponding to at least one target user intention;

[0082] Step S1233, determining a second access parameter corresponding to the voice function point based on the first access parameter, wherein the voice function point is associated with at least one target user intention, and the second access parameter is used to represent the number of independent accesses corresponding to the voice function point;

[0083] Step S1234: determining the function point distribution statistics result according to the second access parameter.

[0084] The above target user intent refers to understanding the specific needs of users when interacting with the cockpit system through voice commands. For example, users may control the air conditioner through different expressions, and the intent involved can be classified into "air conditioner temperature adjustment", "air conditioner wind speed adjustment", "air conditioner switch", etc.

[0085] The above-mentioned unique visitor number (UV) refers to the number of users who uniquely visit or use a certain function point within a specific time frame. In online service statistics, UV is an important indicator used to evaluate the frequency of use and popularity of functions. In the scenario of a smart cockpit, UV can reflect how many different vehicles have issued a specific voice command or request, thereby helping to evaluate the user coverage and representativeness of a function point.

[0086] The above-mentioned function point distribution statistics include the usage frequency and weight of each function point. Among them, the usage frequency refers to the number of times or frequency that a specific function point is triggered by users in the data source, reflecting the actual use of the function by users; the weight calculation is based on the UV aggregation of the user intent dimension. Function points with high UVs indicate that more different users have triggered the same command or similar commands, indicating that the function point has a wider coverage among users and is more representative.

[0087] For example, taking air conditioning as an example, the intents involved in air conditioning function points include but are not limited to "air conditioning temperature adjustment", "air conditioning wind speed adjustment", "air conditioning switch", etc., and each intent includes various user statements, each user statement has a UV value, and the UV values ​​of user statements under the same intent are aggregated to obtain the UV value of the intent. Similarly, the UV of each function point can be obtained. Assuming that the UV of each function point is a, the usage frequency f and weight W of each function point can be calculated by the following formula:

[0088]

[0089] Based on the above optional embodiments, at least one target user intention is determined based on the normalization processing result, and the first access parameter and the second access parameter corresponding to the at least one target user intention are obtained. Finally, the function point distribution statistical result is determined according to the second access parameter. This can construct a test set that is closer to the user's actual usage scenario, has a balanced distribution of function points, and has high data quality, thereby significantly improving the test effect and optimization efficiency of the smart cockpit voice model.

[0090] Optionally, in step S12, mining is performed based on the user behavior data to obtain an initial test data set, which also includes: filtering the user behavior data using domain intent association rules to obtain a second filtering result, wherein the second filtering result is used to represent invalid intent data in the user behavior data; determining a first keyword based on the first filtering result, and determining a second keyword based on the second filtering result; determining a supplementary test data set based on the first keyword and the second keyword, wherein the supplementary test data set is used to supplement the tuning test data set.

[0091] The first keyword mentioned above refers to the effective keyword that frequently appears in user instructions and is closely related to specific fields and user intentions. Effective keywords can accurately reflect the real needs of users and are an important part of building a high-quality test set.

[0092] The second keyword mentioned above refers to the keyword with a frequency greater than or equal to 2 in the invalid intent data set. Although the second keyword has a low correlation with the function point intent, it may reveal the user's uncovered needs or potential problem points of the model, so it still has reference value in test set construction.

[0093] For example, taking the construction of the test set for the air conditioning control function as an example, it is assumed that the smart cockpit voice model focuses on core intents such as "temperature adjustment", "wind speed control" and "mode switching" when processing user instructions about air conditioning. First, the user behavior data is screened in the first round to filter out user instructions closely related to the core intent, and the first keywords, such as "temperature", "wind speed", etc., are extracted from them; secondly, the data in the data source that are not filtered by the intent are not without practical meaning. These data may just not be recalled by the model. Secondary screening can be performed through keywords to filter out keywords with a frequency greater than or equal to 2 in the invalid intent data, such as "air freshness", "dehumidification", etc.; finally, the supplementary test data set is determined based on the first and second keywords, thereby constructing a more comprehensive smart cockpit voice model test set.

[0094] Based on the above optional embodiments, the user behavior data is filtered and processed by utilizing domain intent association rules to obtain a second filtering result, and the first keyword is determined based on the first filtering result, and then the second keyword is determined based on the second filtering result, and finally a supplementary test data set is determined based on the first keyword and the second keyword, which not only covers high-frequency and effective user intentions, but also increases the coverage of low-frequency and potential user intentions, thereby improving the comprehensiveness and accuracy of the test set.

[0095] Optionally, determining the first keyword based on the first screening result includes: obtaining word frequencies and inverse document frequencies corresponding to multiple candidate keywords in the first screening result; and determining the first keyword based on the word frequencies and inverse document frequencies.

[0096] The above term frequency (TF) refers to the frequency of a word appearing in a document, that is, the ratio of the number of times the word appears in a document to the total number of words in the document. In text analysis and information retrieval, TF is an indicator to measure the importance of a word in a document. The higher the TF, the more times the word appears in the document.

[0097] The above-mentioned Inverse Document Frequency (IDF) is a supplement to word frequency and is used to measure the importance of a word. The higher the IDF value, the fewer documents the word appears in the document collection, the higher the word's discrimination, and the stronger its representativeness to the document; conversely, if a word appears in a large number of documents, its IDF is low, indicating that the word's discrimination is not high.

[0098] The above term frequency and inverse document frequency (TF-IDF) is a commonly used text feature extraction method. By multiplying the TF value and IDF value of each word, the TF-IDF value of the word is obtained to reflect the importance of a word in a specific document. When the TF-IDF value of a word is high, it means that the word not only appears frequently in the document in which it is located, but also is sparsely distributed in the entire document collection, thereby highlighting the characteristics of the document. Therefore, words with high TF-IDF values ​​are more suitable as key features of documents and are used in scenarios such as text indexing, classification or information retrieval.

[0099] Figure 4 is a schematic diagram of a supplementary test set construction process according to one embodiment of the present disclosure, such as Figure 4As shown in the figure, when constructing the supplementary test set, the original data is first screened and divided into a valid intent data set and an invalid intent data set according to whether it hits the valid intent. For the valid intent data set, its TF, IDF, and TF-IDF are calculated respectively, and the keywords are aggregated and unified, and the keywords with a frequency of 1 are filtered to ensure that the selected keywords are representative and universal, so as to extract keywords based on the TF-IDF algorithm and construct a valid keyword data set; for the invalid intent data set, keywords with a frequency of less than 2 are discarded, and keywords with a frequency of greater than or equal to 2 are combined with the valid keyword data set as a supplementary test set.

[0100] Based on the above optional embodiments, the word frequency and inverse document frequency corresponding to multiple candidate keywords in the first screening result are obtained, and the first keyword is determined based on the word frequency and inverse document frequency. This can effectively improve the data quality of the test set and ensure that the test of the smart cockpit speech model is closer to the real user scenario, thereby improving the accuracy of the evaluation and optimization of the model performance.

[0101] Optionally, in step S13, the initial test data set is optimized to obtain the optimized test data set, including:

[0102] Step S131, acquiring a first function point and a second function point based on an initial test data set, wherein the number of independent accesses corresponding to the first function point is greater than a preset threshold, and the number of independent accesses corresponding to the second function point is less than the preset threshold;

[0103] Step S132, performing peak clipping processing on the first ratio corresponding to the first function point to obtain a first adjustment result;

[0104] Step S133, performing supplementary processing on the second ratio corresponding to the second functional point to obtain a second adjustment result;

[0105] Step S134: determine an optimized test data set using the first adjustment result and the second adjustment result.

[0106] The first function point mentioned above is the high-frequency function point, which refers to the function point that is frequently used by users in the smart cockpit voice interaction and has a variety of user expressions. Due to the high frequency of use, this type of function point tends to occupy a large proportion in the test set, resulting in an imbalance in the proportion of function points.

[0107] The second function point mentioned above is the low-frequency function point, which refers to a function point that is used relatively infrequently but is important. Although such function points are not common in daily use, they may be crucial to the user experience in special scenarios.

[0108] The peak clipping process mentioned above is the peak clipping of high-frequency function points, which refers to adjusting the proportion of high-frequency function points in the test set, extracting representative statements and controlling their proportion in the test set to avoid excessive impact on the test results. The specific operation is to select high-frequency header data with a difference rate of no more than 10% based on the UV of the user's instruction as the representative statement of the high-frequency function point.

[0109] The above-mentioned supplementary processing, namely the supplementation of low-frequency function point statements, refers to the artificial addition of generalized statements for low-frequency but important function points to increase their coverage in the test set and ensure that the test set can comprehensively evaluate the performance of the speech model. For function points with little generalization significance, low frequency of use and low importance, no additional data expansion is required.

[0110] For example, it is assumed that the initial test data set contains 10,000 user instructions extracted from the user's online log, the preset threshold is 1,000, the UV of the "navigation" function is 2,500, and the UV of "emergency rescue" is 80. According to the preset threshold, it can be determined that "navigation" is a high-frequency function point and "emergency rescue" is a low-frequency function point. For the high-frequency function point of "navigation", the original data is subjected to peak clipping. Specifically, the head data with a difference rate of no more than 10% is screened out from the data with a UV of 2,500. For example, there are 1,500 navigation instructions with a difference rate of less than 10%, and these instructions are regarded as representative statements. By controlling its proportion in the test set, for example, adjusting this part of the data to 60% of the original proportion, that is, 900, to ensure sufficient testing of other function points. For the low-frequency but important function point of "emergency rescue", it has an important impact on the user experience in special scenarios. Therefore, generalized statements are artificially added to improve its coverage. For example, the original test set contained only 80 "emergency rescue" related instructions. Through manual analysis, an additional 200 generalized statements were added, bringing the total number of instructions for this function point to 280. Through the above adjustment process, the proportion of "navigation" function points was adjusted from the original 25% to 9%, while the proportion of "emergency rescue" function points was increased from the original 0.8% to 2.8%, making the overall function point proportion of the test set close to the actual usage ratio, enhancing the representativeness and balance of the test set, and improving the testing and optimization effect of the smart cockpit voice model.

[0111] Based on the above optional embodiments, by tuning the test data set to obtain a tuned test data set, it is possible to ensure that the proportion of high-frequency functional points in the test set is more reasonable and low-frequency functional points are fully covered, thereby enhancing the representativeness and balance of the test set.

[0112] Optionally, in step S14, the tuning test data set is verified, and the quality verification result obtained includes:

[0113] Step S141, performing a first verification process on the tuning test data set to obtain a first verification result, wherein the first verification result is used to represent a labeling verification result corresponding to the tuning test data set;

[0114] Step S142, performing a second verification process on the tuning test data set to obtain a second verification result, wherein the second verification result is used to indicate the degree of distribution closeness between the tuning test data set and the user behavior data;

[0115] Step S143: determining a quality verification result based on the first verification result and the second verification result.

[0116] The first verification process mentioned above refers to manual labeling and secondary quality inspection of the tuning test data set. Manual labeling is the first step in data quality acceptance. Its main purpose is to review the intention and field of model recall and correct possible errors in the model to improve the accuracy of the test data. After labeling, a secondary quality inspection is required to count the data labeling accuracy in each field to ensure that the data quality meets the predetermined standards.

[0117] The above-mentioned second verification process includes but is not limited to the chi-square test, feature cross-validation and JS divergence (Jensen-Shannon Divergence), which is used to measure the distribution proximity between the test set and the online data. The second verification process is a key part of the test set quality acceptance, ensuring that the optimization direction of the model is consistent with the actual user experience to improve the accuracy and effectiveness of the test. Through the second verification process, the test set can be further screened and optimized to make it closer to the user's actual usage scenario, thereby improving the performance of the voice model and user satisfaction.

[0118] Exemplarily, the JS divergence is used to measure the distribution proximity between the test set and the online data. JS divergence combines the advantages of KL divergence (Kullback-Leibler Divergence), and can measure the similarity of two probability distributions symmetrically, avoiding the asymmetric limitations of KL divergence. The smaller the JS divergence value, the closer the two distributions are. KL divergence, also known as relative entropy, is used to measure the degree of difference between two probability distributions. It is a method to describe the difference between two probability distributions P and Q. The smaller the KL divergence value, the closer the distribution is, and the value range is [0,+∞). The KL divergence calculation formula is as follows: where P is the online data set distribution and Q is the test set distribution.

[0119]

[0120] The JS divergence calculation formula is as follows:

[0121]

[0122] Based on the above calculation formula, the degree of closeness between the test set before tuning and the test set after tuning and the online data distribution can be calculated respectively, so as to quantify the difference between the test set and the online data, and then select the best peak shaving strategy to achieve a result that is close to the actual user ratio and meets the peak shaving processing goal.

[0123] Based on the above optional embodiments, by performing verification processing on the tuned test data set to obtain quality verification results, a more accurate, comprehensive and balanced test set can be constructed, thereby improving the performance evaluation and optimization efficiency of the smart cockpit speech model.

[0124] Figure 5 is a schematic diagram of a test set construction method for an intelligent cockpit speech model according to one embodiment of the present disclosure, such as Figure 5 As shown in the figure, in the process of constructing the test set of the intelligent cockpit speech model, first determine whether the online data source belongs to the function point range, and determine the data within the function point range as the valid intent data set, and determine the data outside the function point range as the invalid intent data set. For the valid intent data set, it is necessary to tune the mining results, adjust the function point ratio by supplementing the low-frequency function point statement and cutting the high-frequency function point peak, obtain the optimized test data set, and obtain the top K words with the highest weight based on the word frequency-inverse document frequency algorithm, so as to obtain the effective keywords. In addition, for the valid intent data set, sentence normalization can be performed, and its UV can be aggregated to determine the high / low frequency function points, and then adjust the function point ratio based on the function point distribution results to obtain the optimized test data set. For the invalid intent data set, supplement the mining of its keyword data, and combine the above-mentioned valid keywords and the tuned test data set to obtain the data mining results. Finally, through manual annotation, secondary quality inspection and the closeness of data distribution, it is judged whether the acceptance meets the standards. If the data quality acceptance meets the standards, the current data set is used as the target test set data. Otherwise, the quality data acceptance is performed again after re-tuning until a data set that meets the test requirements is obtained. This disclosure generates a test set that conforms to the actual user usage ratio by deeply mining the user's online logs and optimizing the proportion of functional points. By determining the data quality acceptance plan, the representativeness and quality of the test data are ensured, thereby achieving the technical effect of balancing the functional proportion of the test set and improving data quality.

[0125] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0126] In the embodiments of the present disclosure, a test set construction device for an intelligent cockpit speech model is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the details that have been explained will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0127] Figure 6 is a structural block diagram of a test set construction device for an intelligent cockpit speech model according to one embodiment of the present disclosure, such as Figure 6 As shown, the device comprises:

[0128] An acquisition module 601 is used to acquire user behavior data of a target vehicle, wherein the user behavior data is used to represent voice interaction data of the target vehicle within a preset time period;

[0129] A mining module 602 is used to perform mining processing based on the user behavior data to obtain an initial test data set;

[0130] The tuning module 603 is used to perform tuning processing on the initial test data set to obtain a tuned test data set, wherein the tuned test data is used to represent the function point ratio adjustment result corresponding to the initial test data set;

[0131] A verification module 604 is used to perform verification processing on the tuning test data set to obtain a quality verification result, wherein the quality verification result is used to determine whether the tuning test data set meets a preset acceptance standard;

[0132] The determination module 605 is used to determine a target test data set using the quality verification result, wherein the target test data set is used to perform a functional coverage test on the smart cockpit speech model.

[0133] Optionally, the mining module 602 is also used to: use domain intent association rules to filter user behavior data to obtain a first filtering result, wherein the first filtering result is used to represent valid intent data in the user behavior data; normalize the first filtering result to obtain a normalized result, wherein the normalized result is used to represent a sentence normalization result corresponding to the first filtering result; aggregate the normalized result according to the user intent dimension to obtain a function point distribution statistical result; and determine an initial test data set based on the function point distribution statistical result.

[0134] Optionally, the mining module 602 is also used to: perform data identification on the first screening result to obtain key data information, wherein the key data information includes key sentence information and keyword information corresponding to the first screening result; perform sentence normalization processing using the key data information to obtain a normalized processing result.

[0135] Optionally, the mining module 602 is also used to: determine at least one target user intention based on the normalization processing result; obtain a first access parameter corresponding to at least one target user intention, wherein the first access parameter is used to represent the number of independent accesses corresponding to at least one target user intention; determine a second access parameter corresponding to a voice function point based on the first access parameter, wherein the voice function point is associated with at least one target user intention, and the second access parameter is used to represent the number of independent accesses corresponding to the voice function point; determine the function point distribution statistics according to the second access parameter.

[0136] Optionally, the mining module 602 is also used to: use domain intent association rules to filter and process user behavior data to obtain a second filtering result, wherein the second filtering result is used to represent invalid intent data in the user behavior data; determine a first keyword based on the first filtering result, and determine a second keyword based on the second filtering result; determine a supplementary test data set based on the first keyword and the second keyword, wherein the supplementary test data set is used to supplement the tuning test data set.

[0137] Optionally, the mining module 602 is further used to: obtain word frequencies and inverse document frequencies corresponding to multiple candidate keywords in the first screening result; and determine the first keyword based on the word frequencies and inverse document frequencies.

[0138] Optionally, the tuning module 603 is also used to: obtain a first function point and a second function point based on an initial test data set, wherein the number of independent accesses corresponding to the first function point is greater than a preset threshold, and the number of independent accesses corresponding to the second function point is less than the preset threshold; perform peak shaving processing on a first ratio corresponding to the first function point to obtain a first adjustment result; perform supplementary processing on a second ratio corresponding to the second function point to obtain a second adjustment result; and determine a tuning test data set using the first adjustment result and the second adjustment result.

[0139] Optionally, the verification module 604 is also used to: perform a first verification process on the tuned test data set to obtain a first verification result, wherein the first verification result is used to represent the annotation verification result corresponding to the tuned test data set; perform a second verification process on the tuned test data set to obtain a second verification result, wherein the second verification result is used to represent the degree of distribution proximity between the tuned test data set and the user behavior data; and determine the quality verification result based on the first verification result and the second verification result.

[0140] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0141] According to one embodiment of the present disclosure, there is also provided an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute instructions to implement the test set construction method for the smart cockpit speech model in the embodiment of the present disclosure.

[0142] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0143] S1, obtaining user behavior data of a target vehicle, wherein the user behavior data is used to represent voice interaction data of the target vehicle within a preset time period;

[0144] S2, mining and processing based on user behavior data to obtain the initial test data set;

[0145] S3, performing tuning processing on the initial test data set to obtain a tuned test data set, wherein the tuned test data is used to represent a function point ratio adjustment result corresponding to the initial test data set;

[0146] S4, verifying the tuning test data set to obtain a quality verification result, wherein the quality verification result is used to determine whether the tuning test data set meets a preset acceptance standard;

[0147] S5. Determine a target test data set using the quality verification result, wherein the target test data set is used to perform functional coverage testing on the smart cockpit speech model.

[0148] According to one embodiment of the present disclosure, a computer-readable storage medium is also provided, which includes a stored executable program, wherein when the executable program is running, the device where the storage medium is located is controlled to execute the test set construction method for the smart cockpit speech model in the embodiment of the present disclosure.

[0149] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0150] S1, obtaining user behavior data of a target vehicle, wherein the user behavior data is used to represent voice interaction data of the target vehicle within a preset time period;

[0151] S2, mining and processing based on user behavior data to obtain the initial test data set;

[0152] S3, performing tuning processing on the initial test data set to obtain a tuned test data set, wherein the tuned test data is used to represent a function point ratio adjustment result corresponding to the initial test data set;

[0153] S4, verifying the tuning test data set to obtain a quality verification result, wherein the quality verification result is used to determine whether the tuning test data set meets a preset acceptance standard;

[0154] S5. Determine a target test data set using the quality verification result, wherein the target test data set is used to perform functional coverage testing on the smart cockpit speech model.

[0155] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0156] According to one embodiment of the present disclosure, a computer program product is also provided, including a computer program, which, when executed by a processor, implements the test set construction method for the smart cockpit speech model in the embodiment of the present disclosure.

[0157] Optionally, in this embodiment, the computer program product may be configured as a computer program for executing the following steps:

[0158] S1, obtaining user behavior data of a target vehicle, wherein the user behavior data is used to represent voice interaction data of the target vehicle within a preset time period;

[0159] S2, mining and processing based on user behavior data to obtain the initial test data set;

[0160] S3, performing tuning processing on the initial test data set to obtain a tuned test data set, wherein the tuned test data is used to represent a function point ratio adjustment result corresponding to the initial test data set;

[0161] S4, verifying the tuning test data set to obtain a quality verification result, wherein the quality verification result is used to determine whether the tuning test data set meets a preset acceptance standard;

[0162] S5. Determine a target test data set using the quality verification result, wherein the target test data set is used to perform functional coverage testing on the smart cockpit speech model.

[0163] In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0164] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0165] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0166] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0167] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0168] The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present disclosure. These improvements and modifications should also be regarded as the scope of protection of the present disclosure.

Claims

1. A test set construction method for an intelligent cockpit speech model, characterized in that: include: Acquire user behavior data of a target vehicle, wherein the user behavior data is used to represent voice interaction data of the target vehicle within a preset time period; Perform mining processing based on the user behavior data to obtain an initial test data set; Performing tuning processing on the initial test data set to obtain a tuned test data set, wherein the tuned test data is used to represent a function point ratio adjustment result corresponding to the initial test data set; Performing verification processing on the tuning test data set to obtain a quality verification result, wherein the quality verification result is used to determine whether the tuning test data set meets a preset acceptance standard; A target test data set is determined using the quality verification result, wherein the target test data set is used to perform functional coverage testing on the smart cockpit speech model.

2. The method according to claim 1, characterized in that The initial test data set is obtained by performing mining processing based on the user behavior data, including: Using the domain intention association rule to filter the user behavior data to obtain a first filtering result, wherein the first filtering result is used to represent valid intention data in the user behavior data; Performing normalization processing on the first screening result to obtain a normalized processing result, wherein the normalized processing result is used to represent a sentence normalization result corresponding to the first screening result; Aggregating the normalized processing results according to the user intention dimension to obtain a function point distribution statistical result; The initial test data set is determined based on the function point distribution statistical result.

3. The method according to claim 2, characterized in that The first screening result is normalized to obtain the normalized result including: Performing data identification on the first screening result to obtain key data information, wherein the key data information includes key sentence information and keyword information corresponding to the first screening result; The key data information is used to perform sentence normalization processing to obtain the normalization processing result.

4. The method according to claim 2, characterized in that: The normalization processing result is aggregated according to the user intention dimension to obtain the function point distribution statistical result including: Determining at least one target user intention based on the normalization processing result; Acquire a first access parameter corresponding to the at least one target user intention, wherein the first access parameter is used to indicate the number of independent accesses corresponding to the at least one target user intention; Determining a second access parameter corresponding to a voice function point based on the first access parameter, wherein the voice function point is associated with the at least one target user intention, and the second access parameter is used to indicate the number of independent accesses corresponding to the voice function point; The function point distribution statistics result is determined according to the second access parameter.

5. The method according to claim 2, characterized in that: The method further comprises: Using the domain intention association rule to filter the user behavior data to obtain a second filtering result, wherein the second filtering result is used to represent invalid intention data in the user behavior data; determining a first keyword based on the first screening result, and determining a second keyword based on the second screening result; A supplementary test data set is determined based on the first keyword and the second keyword, wherein the supplementary test data set is used to supplement the tuning test data set.

6. The method according to claim 5, characterized in that Determining the first keyword based on the first screening result includes: Obtaining word frequencies and inverse document frequencies corresponding to a plurality of candidate keywords in the first screening result; The first keyword is determined based on the word frequency and the inverse document frequency.

7. The method according to claim 1, characterized in that The initial test data set is tuned to obtain the tuned test data set, including: Acquire a first function point and a second function point based on the initial test data set, wherein the number of independent accesses corresponding to the first function point is greater than a preset threshold, and the number of independent accesses corresponding to the second function point is less than the preset threshold; Performing peak clipping processing on a first ratio corresponding to the first function point to obtain a first adjustment result; Performing supplementary processing on the second ratio corresponding to the second functional point to obtain a second adjustment result; The tuning test data set is determined using the first adjustment result and the second adjustment result.

8. The method according to claim 1, characterized in that The tuning test data set is verified to obtain the quality verification result, including: Performing a first verification process on the tuned test data set to obtain a first verification result, wherein the first verification result is used to represent a label verification result corresponding to the tuned test data set; Performing a second verification process on the tuning test data set to obtain a second verification result, wherein the second verification result is used to indicate the degree of distribution closeness between the tuning test data set and the user behavior data; The quality verification result is determined based on the first verification result and the second verification result.

9. A test set construction device for an intelligent cockpit speech model, characterized in that: include: An acquisition module, used to acquire user behavior data of a target vehicle, wherein the user behavior data is used to represent voice interaction data of the target vehicle within a preset time period; A mining module, used to perform mining processing based on the user behavior data to obtain an initial test data set; A tuning module, used for performing tuning processing on the initial test data set to obtain a tuned test data set, wherein the tuned test data is used to represent a function point ratio adjustment result corresponding to the initial test data set; A verification module, used to perform verification processing on the tuning test data set to obtain a quality verification result, wherein the quality verification result is used to determine whether the tuning test data set meets a preset acceptance standard; A determination module is used to determine a target test data set using the quality verification result, wherein the target test data set is used to perform functional coverage testing on the smart cockpit speech model.

10. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the test set construction method for the smart cockpit speech model described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the storage medium is located is controlled to execute the test set construction method for the smart cockpit speech model described in any one of claims 1 to 8.

12. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the test set construction method for the intelligent cockpit speech model according to any one of claims 1 to 8.

Images