A large model-based human-computer voice precise interaction method

Abstract

The present application belongs to the technical field of voice interaction, and particularly relates to a human-machine voice precise interaction method based on a large model. In view of the problems of voice source confusion, low instruction recognition rate and insufficient interaction safety caused by the complex acoustic environment in the vehicle cabin, the present application fuses reverberation features and harmonic attenuation features to construct acoustic fingerprints, and realizes high-precision sound source positioning in combination with seat occupancy state verification; performs priority filtering and semantic pre-alignment on non-main driver voice, and eliminates fragmented interference; introduces a large language model to standardize the mapping of structured text into instructions conforming to syntax, format and semantic specifications, and embeds a dynamic permission verification mechanism based on sound source position. The scheme significantly improves the voice instruction recognition accuracy and system robustness, effectively suppresses false triggering, enhances user privacy protection, provides an efficient and precise voice interaction experience for the intelligent cabin, and has outstanding technical practical value and industrial application prospects.

Description

Technical Field

[0001] This invention belongs to the field of voice interaction technology, and in particular relates to a method for accurate human-computer voice interaction based on a large model. Background Technology

[0002] With the rapid development of automotive intelligence, in-vehicle human-machine voice interaction has become a core function for enhancing the driving experience. The complex acoustic environment inside the vehicle, and the simultaneous voice input from multiple occupants, pose significant challenges to the accuracy of voice interaction. How to accurately identify valid voice commands and complete interactions under conditions of strong noise interference and multiple sound sources has become a critical issue that in-vehicle voice technology urgently needs to address. Traditional in-vehicle voice interaction methods have many shortcomings. Voice source recognition relies solely on a single acoustic feature, which is prone to misjudgment. Furthermore, traditional methods lack targeted processing for fragmented and repetitive voice segments, resulting in low semantic parsing accuracy. In addition, the insufficient standardization of voice commands and the lack of source-based permission verification mechanisms further reduce the accuracy and security of the interaction, failing to meet the needs of efficient and reliable voice interaction in in-vehicle scenarios. Summary of the Invention

[0003] To address the technical problems existing in the above-mentioned background art, this invention proposes a human-computer voice precise interaction method based on a large model.

[0004] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps:

[0005] S1. Collect voice data inside the carriage, perform a first preprocessing on the voice data, the first preprocessing being the removal of environmental noise; identify the source of the voice data after the first preprocessing.

[0006] S2. Perform a second preprocessing on the first preprocessed speech data based on source identification. The second preprocessing is to simplify the speech data according to the priority of the source.

[0007] S3. Perform preliminary semantic pre-alignment based on the second preprocessed speech data, and process fragmented and repetitive speech segments.

[0008] S4. Based on the processed speech data, the speech content of the speech data is extracted using a large model and the speech content is standardized and transformed to obtain standardized speech commands.

[0009] S5. Based on source recognition and voice commands, determine whether the current command has permission. If permission is granted, proceed with voice interaction; otherwise, prompt that permission is not granted and wait for secondary confirmation from the user.

[0010] Preferably, the first preprocessing in step S1 for removing environmental noise includes:

[0011] Retrieve the preset pure speech baseline feature set for the carriage scene, perform feature matching between the speech data in the carriage and the baseline feature set, and anchor the core speech feature segments in the speech data to be processed that are consistent with the baseline features.

[0012] Calculate the signal residual between the voice data inside the carriage and the core feature segment of the voice, and define the signal residual as the noise component to be removed;

[0013] The suppression intensity is dynamically adjusted based on the energy proportion of the noise component, and the noise suppression operation is performed iteratively from weak to strong. After each round of suppression, the core speech feature segments are retained and not covered, until the energy proportion of the noise component drops below a preset threshold, and the first preprocessed speech data is obtained.

[0014] Preferably, source identification of the preprocessed speech data includes:

[0015] Clean speech samples from different sources within the carriage, at reasonable speech amplitude thresholds, are collected in advance. Speech reverberation features and harmonic attenuation features of each sample are extracted. Based on the distribution range of the two types of features, an acoustic fingerprint template corresponding to each location is constructed.

[0016] Extract reverberation features and harmonic attenuation features of the same dimension from the first preprocessed speech data to generate a feature set of the speech to be recognized.

[0017] Calculate the feature fit between the set of speech features to be identified and the acoustic fingerprint templates at each location, and mark the location with the highest fit as the initial source location;

[0018] The speech data is subjected to continuous multi-frame feature extraction and fit calculation. A threshold number of frames is set. When the occurrence frequency of the initial source position in the continuous threshold number of frames meets the condition, the position is locked as a candidate source position.

[0019] The system retrieves vehicle seat occupancy status monitoring data. If the seat corresponding to the candidate source location is occupied, the location is confirmed as the voice source location. If the seat is not occupied, the feature fit is recalculated and the set threshold frame number is increased. The source location determination is performed again until the location is confirmed.

[0020] Preferably, step S2 performs a second preprocessing on the first preprocessed speech data based on source identification. The second preprocessing involves simplifying the speech data according to the priority of the source, including:

[0021] The system prioritizes voice data sources. For voice data within a fixed time period, it filters and simplifies the data according to the preset priority. If there are voice segments from the driver's position in the voice data, all voice segments from non-driver's positions are directly removed, and only the voice segments from the driver's position are retained. If there are no voice segments from the driver's position in the voice data, the system determines which segments to retain based on the priority and the length of the voice segments from each position.

[0022] The selected and retained audio segments are arranged linearly according to their timestamps to form continuous audio data along the timeline.

[0023] Preferably, the specific implementation of determining and retaining based on priority and speech length at each position is as follows:

[0024] The effective voice duration of each location within a fixed time period is statistically analyzed based on location tags. A minimum effective voice duration threshold is preset, and only locations with a duration greater than or equal to the preset minimum effective voice duration threshold are considered.

[0025] The priority and effective speech duration of each position are normalized, and then a weighted sum is obtained. The position with the highest weighted score is selected as the target retention position for speech determination and retention.

[0026] Preferably, step S3, which performs preliminary semantic pre-alignment based on the second preprocessed speech data and processes fragmented and repetitive speech segments, is specifically implemented as follows:

[0027] The preprocessed speech data is divided into several semantic segments according to natural semantic pauses. Each segment is assigned a unique identifier, and a context window for each segment is constructed, with the window range being one adjacent segment before and after the segment. The core semantic anchor and semantic activity value of each segment are extracted. Meaningless segments with semantic activity values ​​less than a set threshold are removed, and the set of valid segments is retained.

[0028] Extract fragmented segments that meet preset conditions from the set of valid segments, and calculate the correlation between the fragmented segments and the adjacent non-fragmented segments in the context.

[0029] If only a single contextual unfragmented segment meets the relevance threshold, the fragmented segment is integrated into the corresponding semantic position of the unfragmented segment, and two phoneme transition frames are inserted to eliminate splicing temporal gaps; if multiple contextual unfragmented segments meet the relevance threshold, the segment with the closest temporal sequence is selected as the fusion object, and the above fusion operation is performed; if no contextual unfragmented segment meets the relevance threshold, it is deleted, resulting in a fused segment set.

[0030] Based on the contextual anchor relevance in the fragment set, calculate the anchor matching degree between any fragment and other fragments. If the anchor matching degree is greater than or equal to the set threshold, the two fragments are classified into the same repeating cluster.

[0031] For each duplicate cluster, retain the fragment with the highest context window integrity (i.e., the fragment with the largest sum of semantic relevance between adjacent fragments) as the baseline fragment, remove the remaining duplicate fragments, and update the fragment set;

[0032] All segments in the updated set are sorted in ascending order by timestamp, and the phoneme boundaries between adjacent segments are smoothed to form a temporally continuous sequence of speech segments.

[0033] As a preferred embodiment, step S4, based on the processed speech segment, utilizes a large model to extract the speech content from the speech data and performs standardized transformation on the speech content to obtain standardized speech commands, as follows:

[0034] The speech data, after initial semantic pre-alignment and processing of fragmented and repetitive speech segments, is input into a pre-trained large model to obtain structured text.

[0035] Based on a preset voice command specification template, the structured text is standardized and mapped into a standardized voice command that meets the requirements of syntax, format and semantics, and the standardized voice command is output.

[0036] Compared with existing technologies, the advantages and positive effects of this invention are as follows: It employs acoustic fingerprint recognition technology that fuses reverberation and harmonic attenuation features, combined with seat occupancy verification, to achieve high-precision voice source localization; priority-based voice data simplification effectively filters interference from non-driver-side sound sources; innovative preliminary semantic pre-alignment intelligently processes fragmented and repetitive voice segments; large-scale model-based standardized transformation of voice content ensures command consistency; and a source-based permission verification mechanism enhances interaction security. Overall, this invention solves the technical challenges of low accuracy and poor reliability of voice interaction in complex acoustic environments within vehicles, significantly improving voice command recognition rate and interactive experience, and meeting the needs for efficient and reliable voice interaction in in-vehicle scenarios. Attached Figure Description

[0037] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart illustrating a method for precise human-computer voice interaction based on a large model. Detailed Implementation

[0039] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described below in conjunction with the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0040] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.

[0041] In the context of rapid advancements in automotive intelligence, in-vehicle human-machine voice interaction has become a core function for enhancing the driving experience, leading to a continuously rising market demand for its accuracy and reliability. However, the complex acoustic environment within the vehicle and the high-frequency scenarios of multiple occupants speaking simultaneously pose significant challenges to voice interaction technology. This invention proposes a specific implementation process for a human-machine voice precision interaction method based on a large model. This method is applied to in-vehicle intelligent voice interaction systems, addressing the technical pain points of low accuracy and poor security in human-machine voice interaction caused by dynamic noise interference, multiple sound sources, fragmented and repetitive voice segments, non-standard commands, and unclear permissions within the vehicle cabin. It achieves efficient and accurate human-machine voice interaction in in-vehicle scenarios, significantly improving voice command recognition rate and interaction reliability. The specific implementation process is as follows: Figure 1 As shown.

[0042] First, voice data inside the carriage is collected, and the voice data undergoes a first preprocessing step, which involves removing environmental noise.

[0043] Further, the first preprocessing for removing environmental noise includes: retrieving a preset clean speech benchmark feature set for a train carriage scene; performing feature matching between the in-carriage speech data and the benchmark feature set; anchoring the core speech feature segments in the speech data to be processed that are consistent with the benchmark features; calculating the signal residual between the in-carriage speech data and the core speech feature segments; defining the signal residual as the noise component to be removed; dynamically adjusting the suppression intensity based on the energy proportion of the noise component; performing noise suppression operations iteratively from weak to strong gradients; retaining the core speech feature segments from being covered after each round of suppression until the energy proportion of the noise component drops below a preset threshold, thus obtaining the speech data after the first preprocessing. Specifically, a clean speech benchmark feature set for a train carriage scene is pre-constructed, which needs to cover clean speech samples under different vehicle speeds and road conditions. The sample extraction dimensions include key acoustic features such as fundamental frequency, formants, and Mel-frequency cepstral coefficients. The collected original speech data from the train carriage is input into the feature matching module, and compared one by one with the benchmark feature set using a cosine similarity algorithm. During the matching process, speech segments with a similarity higher than 85% to the baseline features are selected and anchored as core speech feature segments. These segments are the core of subsequent noise suppression and crucial for ensuring the semantic integrity of the speech. Based on the anchored core speech feature segments, the signal difference between the original carriage speech data and the core feature segments is calculated in the time domain. This difference is the signal residual. The signal residual is directly defined as the noise components to be removed, covering engine noise, tire noise, wind noise, etc. To improve the accuracy of noise definition, auxiliary verification can be performed simultaneously in the frequency domain. Signal components outside the frequency band of the core feature segments are classified as noise supplementary sets, and after merging, a complete noise component dataset is formed. A dynamic gradient iterative noise suppression process is initiated, with the suppression intensity adjusted from weak to strong, and the initial suppression intensity is set to 30%. After each round of suppression operation, a feature verification algorithm is immediately used to confirm that the core speech feature segments have not been covered or distorted. At the same time, the energy proportion of the current noise component is calculated in real time. If the proportion does not drop to a preset threshold (usually set to 5%), the suppression intensity is increased by 10%, and the next iteration begins. Repeat the above process until the energy percentage of noise components is below the threshold. The final output speech data is the clean speech data after the first preprocessing and removal of environmental noise.

[0044] Source identification is performed on the first preprocessed speech data.

[0045] Furthermore, clean speech samples from different sources within the carriage, at reasonable speech amplitude thresholds, are pre-collected. The reverberation and harmonic attenuation features of each sample are extracted, and an acoustic fingerprint template corresponding to each location is constructed based on the distribution range of these two types of features. The extraction of reverberation features involves performing a short-time Fourier transform on each sample to calculate two core indicators: reverberation time (the time required for the speech signal to attenuate by 60dB from a steady state) and the proportion of early reflected sound energy (the ratio of the sum of the energy of the direct speech sound and the energy of the reflected sound in the first 50ms to the total energy). The extraction of harmonic attenuation features involves locating the fundamental frequency sequence of the speech sample using a fundamental frequency detection algorithm, and then extracting the amplitude values ​​of the 1st to 5th harmonics corresponding to the fundamental frequency. The attenuation slope of the harmonic amplitude as the harmonic order increases is calculated; this slope is the core quantitative indicator of the harmonic attenuation feature, and a larger absolute value of the slope indicates faster harmonic attenuation. For all samples at each location, the mean, variance, maximum, and minimum values ​​of reverberation and harmonic attenuation features are calculated to determine the 95% confidence intervals for both types of features, thus defining the effective distribution range of the features at that location. The distribution intervals of the reverberation and harmonic attenuation features at the same location are combined, assigning different weights to the two types of features (e.g., a weight of 0.6 for reverberation and 0.4 for harmonic attenuation), forming a unique acoustic fingerprint template for that location. Template validation is then performed by inputting new clean speech samples to extract features and checking whether the features fall within the distribution interval of the template at the corresponding location. The validation pass rate must reach over 90%; if it fails, additional samples must be added and the feature distribution interval recalculated.

[0046] The reverberation and harmonic attenuation features of the same dimension are extracted from the preprocessed speech data to generate a feature set for the speech to be recognized. Specifically, the denoised speech data after the first preprocessing is subjected to short-time frame segmentation to adapt to the short-time stationary characteristics of speech. The windowed speech data is then processed in the same way to extract reverberation and harmonic attenuation features. The extracted reverberation feature vector and harmonic attenuation feature vector are aligned in dimension and concatenated into a high-dimensional feature vector in a preset order. Normalization is then performed on this feature vector to eliminate the influence of differences in the dimensions of different features. Finally, the feature vectors of all speech frames are summarized to generate the feature set for the speech to be recognized.

[0047] The feature fit between the set of speech features to be identified and the acoustic fingerprint templates at each location is calculated, and the location with the highest fit is marked as the initial source location. Specifically, the reverberation features of the speech to be identified (including two sub-features: reverberation time and the proportion of early reflected sound energy) and harmonic attenuation features (attenuation slope) are integrated into a three-dimensional feature vector. Extract the standard vector of the acoustic fingerprint template at each location: For the acoustic fingerprint template at each location, take the mean of the 95% confidence interval of its reverberation feature and harmonic attenuation feature to construct the standard template vector for that location. Here, i represents the different positions. A min-max normalization method is used to map the values ​​of each dimension of the feature vector to be identified and all template vectors to the [0,1] interval, eliminating the interference of differences in the dimensions of different features on the fit calculation. Weights are assigned to different feature dimensions: the combined weight of the two sub-features of the reverberation feature is 0.6, and the weight of the harmonic attenuation feature is 0.4. The values ​​of each dimension of the feature vector to be identified and the template vectors are multiplied by their corresponding weights to obtain the weighted feature vector. The cosine similarity algorithm is used to calculate the fit between the weighted feature vector to be identified and the template vectors at each position: .in, This represents the degree of fit between the speech to be recognized and the template at the i-th position, and its value ranges from [0,1]. The closer the value is to 1, the higher the feature consistency. These are the weighted feature vectors. The acoustic fingerprint templates for all carriage locations are traversed, and the fit score between the speech to be identified and each location is calculated. The fit scores of all locations are compared, and the location corresponding to the maximum value of Sim(i) is selected and marked as the initial source location. A minimum fit score threshold (0.6) is set. If the maximum value is lower than this threshold, it is determined that there is no valid matching location, and the speech features to be identified need to be extracted again and recalculated.

[0048] The speech data undergoes continuous multi-frame feature extraction and fit calculation. A threshold frame number is set. When the frequency of occurrence of the initial source position in the consecutive threshold frame data meets a certain condition, that position is locked as a candidate source position. Specifically, after obtaining the initial source position of a single frame, the initial source positions of all speech frames are recorded in frame order. A preset threshold frame number N and a minimum occurrence count M are used. Only when the occurrence count of the same initial source position in a consecutive N-frame window is ≥ M can the candidate position locking process begin; otherwise, the window is considered to have no valid source position. A sliding window mechanism is used to traverse all speech frames, with the window length fixed at the threshold frame number N. Starting from the first frame, one frame is slid at a time to generate consecutive frame windows. For the N frames in each window, the frequency of occurrence of each initial source position is counted, and the position with the highest frequency is selected. It is verified whether the frequency of this position meets the condition ≥ M. If it does, the position is directly locked as a candidate source position, and the sliding statistics are terminated; if not, the sliding window continues until all frames have been traversed.

[0049] Finally, the vehicle seat occupancy status monitoring data is retrieved. If the seat corresponding to the candidate source location is occupied, the location is confirmed as the voice source location. If the seat is not occupied, the feature fit is recalculated and the set threshold frame number is increased. The source location determination is performed again until the location is confirmed.

[0050] The speech data after the first preprocessing is subjected to a second preprocessing based on the source identification. The second preprocessing is to simplify the speech data according to the priority of the source.

[0051] Furthermore, a preset priority is set for the source of voice data. For voice data within a fixed time period, voice data is filtered and simplified according to the preset priority. If there are voice segments in the voice data that are from the driver's position, all voice segments that are not from the driver's position are directly removed, and only the voice segments from the driver's position are retained. If there are no voice segments from the driver's position in the voice data, the retention is determined by combining the priority and the length of the voice segments in each position.

[0052] The specific implementation of determining and retaining speech based on priority and the length of speech at each location involves: statistically analyzing the effective speech duration at each location within a fixed time period based on location tags; setting a minimum effective speech duration threshold; and only considering locations with a duration greater than or equal to the minimum threshold. The location tags are obtained after source identification. A fixed statistical time window for in-vehicle voice interaction is preset. Effective speech frames are defined based on the acoustic features of the speech frames, and frames with a signal-to-noise ratio greater than a set value are selected as effective frames. The total duration of effective speech frames within a fixed time period is accumulated according to the location dimension to obtain the initial effective speech duration for each location. Priority weight coefficients for non-driver's positions are preset, and weight coefficients are assigned sequentially from highest to lowest priority to the passenger seat, rear left seat, and rear right seat. The passenger seat weight coefficient is assigned 0.6, the rear left seat weight coefficient is assigned 0.2, and the rear right seat weight coefficient is assigned 0.2. The effective speech duration of each non-driver's position speech segment is statistically analyzed. Calculate the weighted composite score for each non-driver's position, normalize the priority and effective speech duration of each position, and then perform a weighted sum to obtain the weighted composite score; Weighted composite score = Normalized priority weight coefficient × Normalized effective speech duration. The position with the highest weighted composite score is selected as the target retention position for speech determination and retention.

[0053] The selected and retained audio segments are arranged linearly according to their timestamps to form continuous audio data along the timeline.

[0054] Then, preliminary semantic pre-alignment is performed based on the second preprocessed speech data, and fragmented and repetitive speech segments are processed.

[0055] Furthermore, the preprocessed speech data is segmented into several semantic segments based on natural semantic pauses, and each segment is assigned a unique identifier. Simultaneously, a context window is constructed for each segment, with the window encompassing one adjacent segment before and after the segment; the core semantic anchor points of each segment are extracted. and semantic activity value Remove meaningless segments with semantic activity values ​​below a set threshold, and retain the set of valid segments.

[0056] Furthermore, the core semantic anchor point The system consists of noun-based operation objects and verb-based operation actions in in-vehicle interaction scenarios. End-to-end speech-to-text (ASR) is performed on the pre-processed speech segment to generate a corresponding text sequence. Then, a vehicle-specific domain semantic slot library (covering operation actions, objects, and parameter slots for high-frequency scenarios such as navigation, air conditioning, and entertainment) is invoked. Through rule matching combined with a lightweight Transformer semantic parsing model, the two core semantic elements—verbal operation actions and noun-based operation objects—are accurately identified from the text sequence, filtering out redundant interjections, filler words, and other non-interactive information. Next, the identified core semantic elements are mapped to a unique encoding system in the vehicle-specific knowledge base, and the encodings are combined to form a structured feature vector, thus obtaining the core semantic anchor points of the segment. Semantic activity value. The calculation method is as follows: ,in, Let i be the effective speech duration of the i-th segment. Total duration. Excluded. Meaningless interfering fragments with a value less than 0.5 are excluded from the set of valid semantic fragments.

[0057] Extract fragmented segments that meet preset conditions from the set of valid segments, and calculate the correlation between the fragmented segments and the adjacent non-fragmented segments in the context.

[0058] Furthermore, a dual threshold for judging fragmented segments is set specifically for in-vehicle scenarios, which will determine the effective segments that meet the criteria. Less than 120ms and Fragments ∈ [0.5, 0.65] are labeled as fragmented fragments. Where k is the fragment number. Calculate With context-unfragmented fragments Relevance: ,in Set a correlation threshold for the number of elements in the set, and perform differentiated fusion based on the correlation results.

[0059] If only a single contextual unfragmented segment meets the relevance threshold, the fragmented segment is integrated into the corresponding semantic position of the unfragmented segment, and two phoneme transition frames are inserted to eliminate splicing temporal gaps; if multiple contextual unfragmented segments meet the relevance threshold, the segment with the closest temporal sequence is selected as the fusion object, and the above fusion operation is performed; if no contextual unfragmented segment meets the relevance threshold, it is deleted, resulting in a fused segment set.

[0060] Based on the contextual anchor relevance in the fragment set, calculate the anchor matching degree between any fragment and other fragments. If the anchor matching degree is greater than or equal to the set threshold, the two fragments are classified into the same repeating cluster.

[0061] Furthermore, the anchor point matching degree is calculated as follows: ,in, These are the timestamps of the i-th and j-th semantic segments, representing the start time of that segment in the speech data. This is the temporal decay coefficient for in-vehicle voice interaction scenarios, used to avoid misjudging non-similar segments across time periods.

[0062] For each duplicate cluster, retain the fragment with the highest context window integrity (i.e., the fragment with the largest sum of semantic relevance between adjacent fragments) as the baseline fragment, remove the remaining duplicate fragments, and update the fragment set;

[0063] All segments in the updated set are sorted in ascending order by timestamp, and the phoneme boundaries between adjacent segments are smoothed to form a temporally continuous sequence of speech segments.

[0064] Next, based on the processed speech data, a large model is used to extract the speech content and standardize it to obtain standardized speech commands. This is achieved by inputting the speech data, after initial semantic pre-alignment and processing of fragmented and repetitive speech segments, into a pre-trained large model to obtain structured text. Based on a pre-defined speech command template, the structured text is standardized and mapped into standardized speech commands that conform to grammatical, format, and semantic requirements, and then the standardized speech commands are output. Specifically, converting speech data into structured text requires integrating Automatic Speech Recognition (ASR) and Large Language Modeling (LLM) technologies to form an efficient two-stage pipeline. First, acoustic features (such as Mel spectrograms) are extracted and input into a pre-trained ASR model (such as Whisper or Wav2Vec 2.0). This type of end-to-end model, through an encoder-decoder architecture, is pre-trained on massive amounts of multilingual speech-text pairs, accurately transcribes the original text, and possesses noise resistance and multi-dialect adaptability. Second, text optimization: punctuation restoration, case correction, and simple error correction are performed on the ASR output to improve text quality. The core step is structured generation: the cleaned text is fed into a pre-trained LLM (such as GPT-4 or Claude), combined with prompt engineering (output in JSON format) or domain fine-tuning. Leveraging the LLM's semantic understanding and instruction compliance capabilities, key entities and relationships are accurately extracted and formatted into the target structure (JSON / XML), resulting in a structured text description. The structured data generated by the LLM is then mapped and transformed according to a pre-defined voice command specification template. This template includes syntactic structure, field format, and semantic requirements to ensure that the output commands meet the needs of the in-vehicle scenario. For example, "turn on the air conditioner" is mapped to a standardized format of {"command":"AC","action":"open","parameter":""}. The system automatically verifies whether the mapping result conforms to the pre-defined syntactic, format, and semantic requirements. If a mismatch is found, it is corrected, and the output response text is converted into a standardized voice command, achieving accurate voice interaction.

[0065] The system determines whether the current command is authorized based on source identification and voice commands. If authorized, voice interaction is initiated; otherwise, an "invalid" message is displayed, and the user is prompted for secondary confirmation. This secondary confirmation involves prompting the user to rephrase their request and then repeating the above steps.

[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments for application in other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for accurate human-computer voice interaction based on a large model, characterized in that, Includes the following steps: S1. Collect voice data inside the carriage, and perform a first preprocessing on the voice data, wherein the first preprocessing is to remove environmental noise; Source identification is performed on the first preprocessed speech data; S2. Perform a second preprocessing on the first preprocessed speech data based on source identification. The second preprocessing is to simplify the speech data according to the priority of the source. S3. Perform preliminary semantic pre-alignment based on the second preprocessed speech data, and process fragmented and repetitive speech segments. S4. Based on the processed speech data, the speech content of the speech data is extracted using a large model and the speech content is standardized and transformed to obtain standardized speech commands. S5. Based on source recognition and voice commands, determine whether the current command has permission. If it has permission, proceed with voice interaction; otherwise, prompt that there is no permission and wait for secondary confirmation from the user. The first preprocessing step S1, which involves removing environmental noise, includes: Retrieve the preset pure speech baseline feature set for the carriage scene, perform feature matching between the speech data in the carriage and the baseline feature set, and anchor the core speech feature segments in the speech data to be processed that are consistent with the baseline features. Calculate the signal residual between the voice data inside the carriage and the core feature segment of the voice, and define the signal residual as the noise component to be removed; The suppression intensity is dynamically adjusted based on the energy proportion of the noise component, and the noise suppression operation is performed iteratively from weak to strong. After each round of suppression, the core speech feature segments are retained and not covered, until the energy proportion of the noise component drops below the preset threshold, and the first preprocessed speech data is obtained. The source identification of the first preprocessed speech data includes: Clean speech samples from different sources within the carriage, at reasonable speech amplitude thresholds, are collected in advance. Speech reverberation features and harmonic attenuation features of each sample are extracted. Based on the distribution range of the two types of features, an acoustic fingerprint template corresponding to each location is constructed. Extract reverberation features and harmonic attenuation features of the same dimension from the first preprocessed speech data to generate a feature set of the speech to be recognized. Calculate the feature fit between the set of speech features to be identified and the acoustic fingerprint templates at each location, and mark the location with the highest fit as the initial source location; The speech data is subjected to continuous multi-frame feature extraction and fit calculation. A threshold number of frames is set. When the occurrence frequency of the initial source position in the continuous threshold number of frames meets the condition, the position is locked as a candidate source position. The system retrieves vehicle seat occupancy status monitoring data. If the seat corresponding to the candidate source location is occupied, the location is confirmed as the voice source location. If the seat is not occupied, the feature fit is recalculated and the set threshold frame number is increased. The source location determination is performed again until the location is confirmed.

2. The method for accurate human-computer voice interaction based on a large model according to claim 1, characterized in that, Step S2 performs a second preprocessing on the first preprocessed speech data based on source identification. The second preprocessing is to simplify the speech data according to the priority of the source, including: The system prioritizes voice data sources. For voice data within a fixed time period, it filters and simplifies the data according to the preset priority. If there are voice segments from the driver's position in the voice data, all voice segments from non-driver's positions are directly removed, and only the voice segments from the driver's position are retained. If there are no voice segments from the driver's position in the voice data, the system determines which segments to retain based on the priority and the length of the voice segments from each position. The selected and retained audio segments are arranged linearly according to their timestamps to form continuous audio data along the timeline.

3. The method for accurate human-computer voice interaction based on a large model according to claim 2, characterized in that, The specific implementation of determining and retaining based on priority and speech length at each position is as follows: The effective voice duration of each location within a fixed time period is statistically analyzed based on location tags. A minimum effective voice duration threshold is preset, and only locations with a duration greater than or equal to the preset minimum effective voice duration threshold are considered. The priority and effective voice duration of each position are normalized, and then a weighted sum is obtained to obtain a weighted comprehensive score. The position with the highest weighted composite score is selected as the target retention position for speech determination and retention.

4. The method for accurate human-computer voice interaction based on a large model according to claim 1, characterized in that, Step S3, which performs preliminary semantic pre-alignment based on the second preprocessed speech data and processes fragmented and repetitive speech segments, is specifically implemented as follows: The preprocessed speech data is divided into several semantic segments according to natural semantic pauses. Each segment is assigned a unique identifier, and a context window for each segment is constructed, with the window range being one adjacent segment before and after the segment. The core semantic anchor and semantic activity value of each segment are extracted. Meaningless segments with semantic activity values ​​less than a set threshold are removed, and the set of valid segments is retained. Extract fragmented segments that meet preset conditions from the set of valid segments, and calculate the correlation between the fragmented segments and the adjacent non-fragmented segments in the context. If only a single contextual unfragmented segment meets the relevance threshold, the fragmented segment is integrated into the corresponding semantic position of the unfragmented segment, and two phoneme transition frames are inserted to eliminate splicing temporal gaps; if multiple contextual unfragmented segments meet the relevance threshold, the segment with the closest temporal sequence is selected as the fusion object, and the above fusion operation is performed; if no contextual unfragmented segment meets the relevance threshold, it is deleted, resulting in a fused segment set. Based on the contextual anchor relevance in the fragment set, calculate the anchor matching degree between any fragment and other fragments. If the anchor matching degree is greater than or equal to the set threshold, the two fragments are classified into the same repeating cluster. For each duplicate cluster, retain the fragment with the highest context window integrity (i.e., the fragment with the largest sum of semantic relevance between adjacent fragments) as the baseline fragment, remove the remaining duplicate fragments, and update the fragment set; All segments in the updated set are sorted in ascending order by timestamp, and the phoneme boundaries between adjacent segments are smoothed to form a temporally continuous sequence of speech segments.

5. The method for accurate human-computer voice interaction based on a large model according to claim 1, characterized in that, Step S4, based on the processed speech segments, uses a large model to extract the speech content from the speech data and standardizes the speech content to obtain standardized speech commands. The speech data, after initial semantic pre-alignment and processing of fragmented and repetitive speech segments, is input into a pre-trained large model to obtain structured text. Based on a preset voice command specification template, the structured text is standardized and mapped into a standardized voice command that meets the requirements of syntax, format and semantics, and the standardized voice command is output.

Images