A vehicle-mounted speech model optimization method and device based on data mining testing

Abstract

The application discloses a kind of based on data mining test vehicle voice model optimization method and device, the method includes: obtaining from the data source of vehicle voice service log system and network crawling system, the data source includes audio data and semantic data;Invalid audio data in the data source is deleted by data screening strategy, and target audio data is obtained;Noise text data and invalid text data in the data source are deleted by data filtering strategy, and target semantic data is obtained;The target audio data and target semantic data are identified, and badcase data is obtained;The badcase data is added to vehicle voice model to avoid learning, to optimize the vehicle voice model.Can continuously optimize model by the mining test of data, and the data of annotation can be used as subsequent model evaluation data, to provide the data set scheme for model major version update iteration replacement evaluation, improve test efficiency and solve model real-time data problem.

Description

Technical Field

[0001] This invention relates to the field of model training technology, specifically to a method and apparatus for optimizing an in-vehicle voice model based on data mining testing. Background Technology

[0002] With the continuous advancement of artificial intelligence and speech recognition technologies, in-vehicle voice assistants have become an important component of intelligent vehicles and an inevitable trend and requirement for the development of automotive intelligence. An in-vehicle voice assistant is a voice interaction system used in automobiles to provide an intelligent interactive experience between the driver and the vehicle, meeting the user's functional and even emotional needs. Through speech recognition and speech synthesis technologies, it enables the driver to control various vehicle functions via voice commands without needing to distract themselves from operating in-vehicle devices or touching screens. In-vehicle voice assistants primarily rely on in-vehicle speech models to recognize the user's voice and interpret their commands.

[0003] Current in-vehicle voice models, such as semantic and ASR models, cannot achieve 100% F1 scores. Therefore, after the model service is launched, data mining and testing processes are needed to continuously optimize the in-vehicle voice model. However, the current in-vehicle voice assistant model lacks a solution for continuous model optimization after launch, making it impossible to guarantee the quantity, real-time nature, and authenticity of user data.

[0004] Therefore, those skilled in the art urgently need to find a new technical solution to address the aforementioned problems. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this invention discloses a method and apparatus for optimizing in-vehicle voice models based on data mining testing.

[0006] According to a first aspect of the disclosed embodiments of the present invention, a method for optimizing an in-vehicle voice model based on data mining testing is provided, the method comprising:

[0007] Acquire data sources from the in-vehicle voice service log system and the web crawling system, the data sources including audio data and semantic data;

[0008] Invalid audio data in the data source is deleted using a data filtering strategy to obtain the target audio data;

[0009] Noisy and invalid text data are removed from the data source using a data filtering strategy to obtain the target semantic data;

[0010] The target audio data and target semantic data are identified to obtain bad case data;

[0011] The bad case data is added to the in-vehicle voice model for avoidance learning, thereby optimizing the in-vehicle voice model.

[0012] Optionally, obtaining data sources from the in-vehicle voice service log system and the web crawling system includes:

[0013] By connecting to the in-vehicle voice service log system through the voice assistant service, the data of service requests are logged;

[0014] Obtain a first data source from the in-vehicle voice service log system, the first data source including the user's actual first audio data and corresponding first semantic data;

[0015] The system periodically crawls real-time film and television media assets, music media assets, and text data from current political and political hot news on the Internet. The text data is second semantic data.

[0016] The second semantic data is converted into second audio data through a timbre synthesis service, and the second semantic data and the second audio data together constitute the second data source.

[0017] Optionally, the step of deleting invalid audio data from the data source and obtaining the target audio data through a data filtering strategy includes:

[0018] The audio data in the data source is segmented into several frames.

[0019] Determine whether the frame data is a silent segment based on the energy spectrum of each frame data;

[0020] The percentage of silent segments in each audio data is counted. If the percentage of silent segments exceeds a preset threshold, the audio data is determined to be invalid.

[0021] Delete invalid audio data and obtain the target audio data.

[0022] Optionally, determining whether a frame data segment is a silent segment based on the energy spectrum of each frame data includes:

[0023] Calculate the energy spectrum of each frame of data, which includes the total energy and spectral entropy index;

[0024] If the total energy of the frame data is lower than a preset energy threshold and the spectral entropy index is lower than a preset index threshold, the frame data will be marked as a silent segment.

[0025] Optionally, the step of deleting noisy and invalid text data from the data source using a data filtering strategy to obtain target semantic data includes:

[0026] Remove garbled characters and special characters from the semantic data to filter out noisy text in the semantic data;

[0027] The semantic data after filtering noisy text is checked for text length. Semantic data with text length greater than a first preset length threshold and text length less than a second preset length threshold are deleted to obtain the target semantic data.

[0028] Optionally, the step of identifying the target audio data and target semantic data to obtain bad case data includes:

[0029] The speech recognition model is used to determine whether the first audio data and the first semantic data are consistent, and inconsistent first audio data and first semantic data are identified as bad case data.

[0030] Determine whether the first semantic data and the second semantic data are within the correct semantic range, and identify the first semantic data and the second semantic data that are not within the correct semantic range as badcase data.

[0031] According to a second aspect of the embodiments disclosed in this invention, an in-vehicle voice model optimization device based on data mining testing is provided, the device comprising:

[0032] The data source acquisition module acquires data sources from the in-vehicle voice service log system and the web crawling system, wherein the data sources include audio data and semantic data.

[0033] The target audio acquisition module is connected to the data source acquisition module. It deletes invalid audio data in the data source through a data filtering strategy and acquires the target audio data.

[0034] The target semantic acquisition module, connected to the target audio acquisition module, deletes noisy text data and invalid text data in the data source through a data filtering strategy to acquire target semantic data;

[0035] The badcase data acquisition module is connected to the target semantic acquisition module, and identifies the target audio data and target semantic data to acquire badcase data;

[0036] The speech model optimization module is connected to the badcase data acquisition module. It adds the badcase data to the vehicle speech model for avoidance learning in order to optimize the vehicle speech model.

[0037] Optionally, the data source acquisition module includes:

[0038] The log storage unit accesses the in-vehicle voice service log system through the voice assistant service to save the data of service requests as logs.

[0039] The first data source acquisition unit is connected to the log storage unit and acquires the first data source from the vehicle voice service log system. The first data source includes the user's real first audio data and the corresponding first semantic data.

[0040] The second semantic data acquisition unit is connected to the first data source acquisition unit and periodically crawls text data from real-time film and television media assets, music media assets, and current political hot news in the network. The text data is the second semantic data.

[0041] The second audio data acquisition unit is connected to the second semantic data acquisition unit. It converts the second semantic data into second audio data through a timbre synthesis service. The second semantic data and the second audio data together constitute the second data source.

[0042] Optionally, the target audio acquisition module includes:

[0043] The segmentation processing unit performs segmentation processing on the audio data in the data source, dividing the audio data into several frame data.

[0044] A silent segment determination unit is connected to the segmentation processing unit and determines whether the frame data is a silent segment based on the energy spectrum of each frame data.

[0045] An invalid audio judgment unit, connected to the silent segment judgment unit, calculates the proportion of silent segments in each audio data. If the proportion of silent segments exceeds a preset proportion threshold, the audio is judged as invalid audio data.

[0046] The target audio acquisition unit is connected to the invalid audio judgment unit, which deletes invalid audio data and acquires target audio data.

[0047] Optionally, the target semantic acquisition module further includes:

[0048] The noisy text filtering unit removes garbled characters and special characters from the semantic data to perform noisy text filtering on the semantic data.

[0049] The target semantic acquisition unit is connected to the noise text filtering unit. It performs text length checks on the semantic data after noise text filtering, deletes semantic data whose text length is greater than a first preset length threshold and whose text length is less than a second preset length threshold, and acquires the target semantic data.

[0050] In summary, this invention discloses a method and apparatus for optimizing an in-vehicle voice model based on data mining testing. The method includes: acquiring data sources from an in-vehicle voice service log system and a web crawling system, the data sources including audio data and semantic data; deleting invalid audio data from the data sources using a data filtering strategy to obtain target audio data; deleting noisy text data and invalid text data from the data sources using a data filtering strategy to obtain target semantic data; identifying the target audio data and target semantic data to obtain bad case data; and adding the bad case data to the in-vehicle voice model for avoidance learning to optimize the in-vehicle voice model. This method enables continuous model optimization through data mining testing, and the labeled data can be used for subsequent model evaluation, providing a dataset scheme for major model version updates and iterations, improving testing efficiency, and solving the problem of real-time model data.

[0051] Other features and advantages disclosed in this invention will be described in detail in the following detailed description section. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0053] Figure 1 This is a flowchart illustrating an in-vehicle voice model optimization method based on data mining testing, according to an exemplary embodiment.

[0054] Figure 2 It is based on Figure 1 A flowchart illustrating a method for obtaining a data source is shown.

[0055] Figure 3 It is based on Figure 1 A flowchart illustrating a method for acquiring target audio data is shown.

[0056] Figure 4 It is based on Figure 1 A flowchart illustrating a method for obtaining bad cases is shown.

[0057] Figure 5 This is a schematic diagram of an NLU data mining testing method;

[0058] like Figure 6 This is an overall flowchart of a method for optimizing in-vehicle voice models based on data mining testing;

[0059] Figure 7 This is a structural block diagram of an in-vehicle voice model optimization device based on data mining testing, according to an exemplary embodiment.

[0060] Figure 8 It is based on Figure 7 The diagram shown is a structural block diagram of a data source acquisition module;

[0061] Figure 9 It is based on Figure 7 The diagram shown is a structural block diagram of a target audio acquisition module;

[0062] Figure 10 It is based on Figure 7 The diagram shows a structural block diagram of a target semantic acquisition module. Detailed Implementation

[0063] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present disclosure.

[0064] Figure 1 This is a flowchart illustrating an in-vehicle voice model optimization method based on data mining testing, according to an exemplary embodiment. Figure 1 As shown, the method includes:

[0065] In step 101, data sources are obtained from the vehicle voice service log system and the web crawling system.

[0066] The data source includes audio data and semantic data.

[0067] For example, the data source for optimizing the in-vehicle voice model is obtained from real user voice data in the in-vehicle voice service log system, ensuring the quantity and authenticity of the test data. Additionally, the data source also includes real-time media asset data from web crawling systems, including real-time film and television assets, music assets, and current political news, to ensure the real-time nature of the test data.

[0068] It is understandable that the audio data and semantic data in the data source are in one-to-one correspondence, including the semantic data obtained by recognizing the audio data, and the audio data obtained after transforming the semantic data.

[0069] Specifically, Figure 2 It is based on Figure 1 The diagram shown illustrates a method for obtaining a data source. Figure 2 As shown, step 101 includes:

[0070] In step 1011, the in-vehicle voice service log system is accessed through the voice assistant service to log the data of service requests.

[0071] In step 1012, a first data source is obtained from the vehicle voice service log system. The first data source includes the user's actual first audio data and the corresponding first semantic data.

[0072] For example, when an online voice assistant service connects to the log system, all data related to service requests will be logged. The logged data includes the user's actual Automatic Speech Recognition (ASR) and semantic data, which is the first data source (first audio data and corresponding first semantic data).

[0073] In step 1013, text data from real-time film and television media assets, music media assets, and current affairs news on the network are crawled periodically. This text data is the second semantic data.

[0074] In step 1014, the second semantic data is converted into second audio data through a timbre synthesis service, and the second semantic data and the second audio data together constitute the second data source.

[0075] For example, since current mainstream models are non-autonomous learning modes, they cannot correctly parse real-time data. To address this issue in the embodiments disclosed in this invention, real-time film and television media assets, music media assets, and current affairs news data are periodically crawled from the internet for testing. This data (i.e., the second semantic data) ensures the real-time nature of the test data. Furthermore, the crawled data is all text data; the text data is converted into ASR data using a sound synthesis service to obtain the second audio data.

[0076] In step 102, invalid audio data in the data source is deleted using a data filtering strategy to obtain the target audio data.

[0077] For example, after obtaining the data source, it needs to be processed to delete invalid data before conducting mining tests, thereby reducing the number of invalid mining tests. Specifically, for audio data, invalid audio data is deleted using a data filtering strategy.

[0078] Specifically, Figure 3 It is based on Figure 1 The diagram illustrates a process for acquiring target audio data, as shown below. Figure 3 As shown, step 102 includes:

[0079] In step 1021, the audio data in the data source is segmented into several frames.

[0080] For example, the audio data is divided into several shorter frames, preferably each frame lasting 20ms-30ms, to perform VAD (Voice Activity Detection) on each frame.

[0081] In step 1022, the energy spectrum of each frame of data is used to determine whether the frame is a silent segment.

[0082] Specifically, the energy spectrum of each frame of data is calculated, which includes the total energy and the spectral entropy index. If the total energy of the frame of data is lower than a preset energy threshold and the spectral entropy index is lower than a preset index threshold, the frame of data is marked as a silent segment.

[0083] For example, the energy spectrum of each frame includes indicators such as total energy and spectral entropy. If the total energy of a frame is lower than a preset energy threshold and the spectral entropy is low, below a preset indicator threshold (indicating no significant speech features), it is marked as a silent segment.

[0084] In step 1023, the proportion of silent segments in each audio data is counted. If the proportion of silent segments exceeds a preset threshold, the audio data is determined to be invalid.

[0085] In step 1024, invalid audio data is deleted and the target audio data is obtained.

[0086] For example, the percentage of silent segments in the entire audio data is calculated. If the percentage of silent segments exceeds a preset threshold (e.g., 50% or higher), the audio is determined to be invalid and the audio data is deleted accordingly.

[0087] In step 103, noisy and invalid text data in the data source are deleted using a data filtering strategy to obtain the target semantic data.

[0088] Specifically, garbled characters and special characters in the semantic data are removed to filter the semantic data for noisy text; the text length of the semantic data after noisy text filtering is checked, and semantic data with a text length greater than a first preset length threshold and a text length less than a second preset length threshold are deleted to obtain the target semantic data.

[0089] For example, semantic data may contain noisy text irrelevant to semantic understanding, such as garbled characters or special characters. Noisy text is removed to filter the semantic data. Additionally, since excessively short text may not contain complete information, while excessively long text may contain redundant or irrelevant information, a preset length threshold is set to remove excessively short or long text (semantic data with a text length greater than a first preset length threshold and a text length less than a second preset length threshold is removed).

[0090] In step 104, the target audio data and target semantic data are identified to obtain badcase data.

[0091] Specifically, Figure 4 It is based on Figure 1 The diagram illustrates a method for obtaining bad cases, as shown below. Figure 4 As shown, step 104 includes:

[0092] In step 1041, the speech recognition model is used to determine whether the first audio data and the first semantic data are consistent, and the inconsistent first audio data and first semantic data are identified as bad case data.

[0093] For example, for the collected first audio data and corresponding first semantic data of the user, the speech recognition model is used to determine whether the first audio data and the first semantic data are consistent. If they are inconsistent, they are determined to be bad case data.

[0094] Alternatively, manual testing can be used. Using an audio playback tool (e.g., Audacity), the correct number of channels and sampling rate are selected, and the relevant audio is listened to manually to determine if it matches the ASR results (the ASR results will associate the audio data with the NLU data in the log system, thus allowing the acquisition of the audio's ASR test results). It should be noted that NLU data (Natural Language Understanding) is semantic data.

[0095] In step 1042, it is determined whether the first semantic data and the second semantic data are in the correct semantic range, and the first semantic data and the second semantic data that are not in the correct semantic range are identified as badcase data.

[0096] For example, determine whether semantic data is within the correct semantic scope; semantic data that is not within the correct semantic scope is considered bad case data. For example... Figure 5As shown, the initial testing of NLU data mining also employs a manual approach. This requires understanding the status and phenomena of all domains, intents, and slots when the NLU model is deployed in a business context, and then manually filtering out data that does not belong to the semantic range. For example, if I want to listen to Jay Chou's songs, the domain is "music," the intent is "play," and the slots are {"player": "Jay Chou"}. If other domain phrases appear in the filtered results for this domain and intent, a false positive occurs, which is considered a bad case. Additionally, there are errors in slot extraction. For instance, if I want to listen to "Singing for the Motherland," and the slot extraction is {"song": "Singing for the Motherland"}, this is also a bad case. Understandably, after this initial manual assessment, subsequent NLU data mining tests can utilize automated processing.

[0097] In step 105, the badcase data is added to the in-vehicle voice model for avoidance learning in order to optimize the in-vehicle voice model.

[0098] like Figure 6 Here is a flowchart of the overall process for optimizing in-vehicle voice models based on data mining testing, as follows: Figure 6 As shown, the processing of bad case data mainly includes two aspects: feeding back into model training and optimization; and labeling to generate answer sets for regression and model validation. Feedback into model training and optimization includes: regularly providing bad cases to model training engineers for training optimization. Labeling to generate answer sets and regression to validate the model includes: manually labeling the ASR and NLU bad case data, labeling incorrect ASR audio data as correct ASR results, and incorrect NLU semantic data as correct domains, intents, and slots. For the next test with the same NLU and ASR data, the labeled test results can be directly compared with the actual test results to generate a difference file. Subsequent labeled test data can then be used for automated testing. For the next batch of user data, deduplication can be performed in advance; if the data has already been labeled, manual testing is unnecessary.

[0099] Additionally, it's worth noting that providing bad cases to model training engineers for optimization can also be automated. Automated processing includes fine-tuning and structural upgrades, pre-processing and post-processing optimization, and the Cringe loss method. Fine-tuning and structural upgrades can be done in two ways: a simple approach and a complex approach. The simple approach involves directly feeding bad cases to the model for learning, using methods like Soft Prompt Tuning (SFT) to allow the large model to learn these error cases, thus quickly improving performance. The complex approach involves observing patterns in bad cases, finding more samples in a targeted manner, and even modifying the model structure (e.g., using LoRa, Adapters, etc.) to make it easier for the model to learn certain information.

[0100] Preprocessing and postprocessing optimization specifically include: Preprocessing: Performing a series of processes on the input data before it enters the model, such as removing punctuation, rewriting, and rejecting incorrect data, to improve the model's processing efficiency. Postprocessing: Reprocessing the model's output results, such as integrating content, applying thresholds after classification, filtering after model recall, and formatting the final returned results, to improve output quality.

[0101] The principle of the CRINGE loss method is to introduce a sequence of negative examples (i.e., completions of prompts that the model should not generate) and train it using the CRINGE (contrast Iterative Negative GEneration) loss function. For positive examples, the maximum likelihood method is used; for negative examples, training is performed by comparing each label in the sequence with one of the language model's predictions. The model is iteratively improved by labeling the model's generation iteratively and applying the CRINGE loss to these examples.

[0102] Figure 7 This is a structural block diagram illustrating an in-vehicle voice model optimization device based on data mining testing, according to an exemplary embodiment. Figure 7 As shown, the device 700 includes:

[0103] The data source acquisition module 710 acquires data sources from the vehicle voice service log system and the web crawling system. These data sources include audio data and semantic data.

[0104] The target audio acquisition module 720 is connected to the data source acquisition module 710. It deletes invalid audio data in the data source through a data filtering strategy and acquires the target audio data.

[0105] The target semantic acquisition module 730, connected to the target audio acquisition module 720, deletes noisy text data and invalid text data in the data source through a data filtering strategy to acquire target semantic data;

[0106] The badcase data acquisition module 740 is connected to the target semantic acquisition module 730. It identifies the target audio data and target semantic data to acquire badcase data.

[0107] The speech model optimization module 750 is connected to the badcase data acquisition module 740. It adds the badcase data to the vehicle speech model for avoidance learning in order to optimize the vehicle speech model.

[0108] Figure 8 It is based on Figure 7 The diagram shown is a structural block diagram of a data source acquisition module, such as... Figure 8 As shown, the data source acquisition module 710 includes:

[0109] The log storage unit 711 accesses the in-vehicle voice service log system through the voice assistant service to save the data of service requests as logs.

[0110] The first data source acquisition unit 712 is connected to the log storage unit 711 and acquires the first data source from the vehicle voice service log system. The first data source includes the user's real first audio data and the corresponding first semantic data.

[0111] The second semantic data acquisition unit 713 is connected to the first data source acquisition unit 712 and periodically crawls text data from real-time film and television media assets, music media assets and current political hot news in the network. The text data is the second semantic data.

[0112] The second audio data acquisition unit 714 is connected to the second semantic data acquisition unit 713. It converts the second semantic data into second audio data through a timbre synthesis service. The second semantic data and the second audio data together constitute the second data source.

[0113] Figure 9 It is based on Figure 7 The diagram shown is a structural block diagram of a target audio acquisition module, such as... Figure 9 As shown, the target audio acquisition module 720 includes:

[0114] The segmentation processing unit 721 performs segmentation processing on the audio data in the data source, dividing the audio data into several frame data.

[0115] The silent segment determination unit 722 is connected to the segmentation processing unit 721, and determines whether the frame data is a silent segment based on the energy spectrum of each frame data.

[0116] The invalid audio judgment unit 723 is connected to the silence segment judgment unit 722. It counts the proportion of silence segments in each audio data. If the proportion of silence segments exceeds a preset proportion threshold, the audio is judged as invalid audio data.

[0117] The target audio acquisition unit 724 is connected to the invalid audio judgment unit 723, which deletes invalid audio data and acquires target audio data.

[0118] Figure 10 It is based on Figure 7 The diagram shown is a structural block diagram of a target semantic acquisition module, such as Figure 10 As shown, the target semantic acquisition module 730 also includes:

[0119] The noisy text filtering unit 731 removes garbled characters and special characters from the semantic data to filter the semantic data from noisy text.

[0120] The target semantic acquisition unit 732 is connected to the noisy text filtering unit 731. It performs text length checks on the semantic data after the noisy text is filtered, deletes semantic data whose text length is greater than a first preset length threshold and whose text length is less than a second preset length threshold, and acquires the target semantic data.

[0121] In summary, this invention discloses a method and apparatus for optimizing an in-vehicle voice model based on data mining testing. The method includes: acquiring data sources from an in-vehicle voice service log system and a web crawling system, the data sources including audio data and semantic data; deleting invalid audio data from the data sources using a data filtering strategy to obtain target audio data; deleting noisy text data and invalid text data from the data sources using a data filtering strategy to obtain target semantic data; identifying the target audio data and target semantic data to obtain bad case data; and adding the bad case data to the in-vehicle voice model for avoidance learning to optimize the in-vehicle voice model. This method enables continuous model optimization through data mining testing, and the labeled data can be used for subsequent model evaluation, providing a dataset scheme for major model version updates and iterations, improving testing efficiency, and solving the problem of real-time model data.

[0122] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0123] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0124] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for optimizing an in-vehicle voice model based on data mining testing, characterized in that, The method includes: Acquire data sources from the in-vehicle voice service log system and the web crawling system, the data sources including audio data and semantic data; Invalid audio data in the data source is deleted using a data filtering strategy to obtain the target audio data; Noisy and invalid text data are removed from the data source using a data filtering strategy to obtain the target semantic data; The target audio data and target semantic data are identified to obtain bad case data; The bad case data is added to the in-vehicle voice model for avoidance learning, so as to optimize the in-vehicle voice model; The acquisition of data sources from the in-vehicle voice service log system and the web crawling system includes: By connecting to the in-vehicle voice service log system through the voice assistant service, the data of service requests are logged; Obtain a first data source from the in-vehicle voice service log system, the first data source including the user's actual first audio data and corresponding first semantic data; The system periodically crawls real-time film and television media assets, music media assets, and text data from current political and political hot news on the Internet. The text data is second semantic data. The second semantic data is converted into second audio data through a timbre synthesis service, and the second semantic data and the second audio data together constitute the second data source.

2. The method for optimizing an in-vehicle voice model based on data mining testing according to claim 1, characterized in that, The step of deleting invalid audio data from the data source and obtaining target audio data through a data filtering strategy includes: The audio data in the data source is segmented into several frames. Determine whether the frame data is a silent segment based on the energy spectrum of each frame data; The percentage of silent segments in each audio data is counted. If the percentage of silent segments exceeds a preset threshold, the audio data is determined to be invalid. Delete invalid audio data and obtain the target audio data.

3. The method for optimizing in-vehicle voice models based on data mining testing according to claim 2, characterized in that, The step of determining whether a frame of data is a silent segment based on the energy spectrum of each frame of data includes: Calculate the energy spectrum of each frame of data, which includes the total energy and spectral entropy index; If the total energy of the frame data is lower than a preset energy threshold and the spectral entropy index is lower than a preset index threshold, the frame data will be marked as a silent segment.

4. The method for optimizing an in-vehicle voice model based on data mining testing according to claim 1, characterized in that, The step of deleting noisy and invalid text data from the data source using a data filtering strategy to obtain target semantic data includes: Remove garbled characters and special characters from the semantic data to filter out noisy text in the semantic data; The semantic data after filtering noisy text is checked for text length. Semantic data with text length greater than a first preset length threshold and text length less than a second preset length threshold are deleted to obtain the target semantic data.

5. The method for optimizing an in-vehicle voice model based on data mining testing according to claim 1, characterized in that, The step of identifying the target audio data and target semantic data to obtain bad case data includes: The speech recognition model is used to determine whether the first audio data and the first semantic data are consistent, and inconsistent first audio data and first semantic data are identified as bad case data. Determine whether the first semantic data and the second semantic data are within the correct semantic range, and identify the first semantic data and the second semantic data that are not within the correct semantic range as badcase data.

6. A vehicle-mounted voice model optimization device based on data mining testing, characterized in that, The device includes: The data source acquisition module acquires data sources from the in-vehicle voice service log system and the web crawling system, wherein the data sources include audio data and semantic data. The target audio acquisition module is connected to the data source acquisition module. It deletes invalid audio data in the data source through a data filtering strategy and acquires the target audio data. The target semantic acquisition module, connected to the target audio acquisition module, deletes noisy text data and invalid text data in the data source through a data filtering strategy to acquire target semantic data; The badcase data acquisition module is connected to the target semantic acquisition module, and identifies the target audio data and target semantic data to acquire badcase data; The speech model optimization module is connected to the badcase data acquisition module. It adds the badcase data to the vehicle speech model for avoidance learning in order to optimize the vehicle speech model. The data source acquisition module includes: The log storage unit accesses the in-vehicle voice service log system through the voice assistant service to save the data of service requests as logs. The first data source acquisition unit is connected to the log storage unit and acquires the first data source from the vehicle voice service log system. The first data source includes the user's real first audio data and the corresponding first semantic data. The second semantic data acquisition unit is connected to the first data source acquisition unit and periodically crawls text data from real-time film and television media assets, music media assets, and current political hot news in the network. The text data is the second semantic data. The second audio data acquisition unit is connected to the second semantic data acquisition unit. It converts the second semantic data into second audio data through a timbre synthesis service. The second semantic data and the second audio data together constitute the second data source.

7. The in-vehicle voice model optimization device based on data mining testing according to claim 6, characterized in that, The target audio acquisition module includes: The segmentation processing unit performs segmentation processing on the audio data in the data source, dividing the audio data into several frame data. A silent segment determination unit is connected to the segmentation processing unit and determines whether the frame data is a silent segment based on the energy spectrum of each frame data. An invalid audio judgment unit, connected to the silent segment judgment unit, calculates the proportion of silent segments in each audio data. If the proportion of silent segments exceeds a preset proportion threshold, the audio is judged as invalid audio data. The target audio acquisition unit is connected to the invalid audio judgment unit, which deletes invalid audio data and acquires target audio data.

8. The in-vehicle voice model optimization device based on data mining testing according to claim 6, characterized in that, The target semantic acquisition module further includes: The noisy text filtering unit removes garbled characters and special characters from the semantic data to perform noisy text filtering on the semantic data. The target semantic acquisition unit is connected to the noise text filtering unit. It performs text length checks on the semantic data after noise text filtering, deletes semantic data whose text length is greater than a first preset length threshold and whose text length is less than a second preset length threshold, and acquires the target semantic data.

Images