Voice processing system and method, vehicle and storage medium

Abstract

The invention relates to the technical field of voice processing, and discloses a voice processing system and method, a vehicle and a storage medium, and the system comprises a recognition module which is used for carrying out the feature recognition of a first audio segment in an audio segment sequence, and obtaining first feature information; performing character recognition based on the first feature information to obtain a first character recognition result; the judgment module is used for fusing the first feature information with second feature information of a previous audio segment of the first audio segment in the audio segment sequence to obtain a fused feature; based on the fusion feature, judging whether the first audio segment and the previous audio segment need to be spliced or not; and the splicing module is used for obtaining the first character recognition result and a second character recognition result corresponding to the previous audio segment when the judgment result is yes, and splicing the first character recognition result and the second character recognition result. The process of obtaining the character recognition result based on the feature information and the process of obtaining the fusion feature or judging whether splicing is carried out are executed in parallel, the voice splicing efficiency can be effectively improved, and semantic response delay is improved.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, specifically to speech processing systems, methods, vehicles, and storage media. Background Technology

[0002] With the development of intelligent connected vehicles, voice interaction has become the mainstream human-computer interaction method in smart cockpits, allowing users to complete multimedia control, navigation planning, and other operations via voice. In current voice interaction methods, a silence duration threshold is set to determine whether to splice two voice segments. For example, when a user says, "Please play… (pause for 3 seconds) A (song name)," although the voice segments before and after the pause express the user's complete request, the silence duration of both segments reaches the silence duration threshold, causing the two segments to be processed as independent requests. Traditionally, extending the silence duration threshold could determine whether to splice the two voice segments, but this reduces the efficiency of voice splicing and increases response latency. Summary of the Invention

[0003] In view of this, embodiments of this application provide a voice processing system, method, vehicle, and storage medium to solve the problems of low voice splicing efficiency and large response delay.

[0004] In a first aspect, embodiments of this application provide a voice processing system, the method comprising: The recognition module is used to perform feature recognition on the first audio segment in the audio segment sequence to obtain first feature information; and to perform character recognition based on the first feature information to obtain a first character recognition result. The judgment module is used to fuse the first feature information with the second feature information of the previous audio segment of the first audio segment in the audio segment sequence to obtain a fused feature; based on the fused feature, it is determined whether the first audio segment and the previous audio segment need to be spliced ​​together. The splicing module is used to obtain the first text recognition result and the second text recognition result corresponding to the previous audio segment when the judgment result is yes, and then splice them together.

[0005] This application embodiment obtains a fusion feature by fusing the second feature information of the first audio segment and the previous audio segment, and improves the accuracy of splicing judgment by using the fusion feature to determine whether splicing should be performed.

[0006] The first text recognition result corresponding to the first audio segment is simultaneously acquired during the process of fusing the second feature information of the first audio segment and the previous audio segment to obtain the fused feature, or during the process of determining whether to splice based on the fused feature. The text acquisition and feature fusion / sponging judgment processes of the audio segment are executed in parallel, without waiting for the fused feature / judgment result before starting text acquisition, or performing feature fusion / sponging judgment only after completing text acquisition. This parallel design enables the synchronous generation of fused feature / sponging results and text output, effectively improving the time consumption caused by serial processing. It eliminates the need to extend the silence duration threshold or train a new model to determine whether to splice the two audio segments, significantly improving speech splicing efficiency and effectively reducing semantic response latency, thus adapting to the high requirements of real-time interaction in smart cockpits.

[0007] When determining the splicing, the pre-stored second character recognition result is integrated with the first character recognition result to ensure that the semantics corresponding to the two segmented speech segments can be completely connected, avoiding command execution failure due to text breakage and improving the reliability of voice interaction.

[0008] The fusion judgment is based on the feature information of existing audio, without the need to introduce additional models or calculation modules to determine the feature information. This improves the accuracy of splicing while avoiding excessive consumption of system resources.

[0009] In conjunction with the first aspect, in an optional implementation, the first feature information includes acoustic features and semantic features of the first audio segment, and the recognition module includes an encoder and a decoder; The encoder is used to receive the first audio segment and perform feature recognition on the first audio segment to obtain the acoustic features of the first audio segment. The decoder is used to receive the acoustic features of the first audio segment and obtain the semantic features of the first audio segment based on the acoustic features of the first audio segment.

[0010] This application embodiment directly reuses the hidden layer outputs of the encoder and decoder of the recognition module as acoustic and semantic features, without the need for an additional feature extraction module to extract feature information. This realizes feature reuse based on the recognition module, which reduces the computational overhead of feature extraction and ensures the correlation between acoustic and semantic features.

[0011] In one optional implementation, the recognition module further includes: a text conversion unit; The text conversion unit is used to perform character recognition based on the acoustic and semantic features of the first audio segment to obtain the first character recognition result.

[0012] In this embodiment, the text conversion unit of the recognition module performs text recognition based on the acoustic features of the first audio segment obtained by the encoder and the semantic features of the first audio segment obtained by the decoder, and obtains the first text recognition result. The judgment module can reuse the first feature information of the first audio segment without affecting the process of the recognition module performing text recognition based on the first feature information of the first audio segment and obtaining the first text recognition result. By executing in parallel, the delay caused by serial processing is avoided, and the immediacy of voice interaction is significantly improved.

[0013] In one optional implementation, the determination module includes a fusion unit; The fusion unit is used for: The acoustic features of the first audio segment and the acoustic features of the previous audio segment are spliced ​​together to obtain spliced ​​acoustic features. The semantic features of the first audio segment and the semantic features of the previous audio segment are concatenated to obtain the concatenated semantic features. From the concatenated semantic features, key features of preset categories are selected to obtain a query matrix; Based on the query matrix, extract at least two types of key acoustic features that are semantically related to the spliced ​​semantic features from the spliced ​​acoustic features; The at least two types of key acoustic features are fused to obtain the fused features.

[0014] This application embodiment extracts key acoustic features based on the query matrix generated by splicing semantic features, realizing the deep fusion of acoustic features and semantic features. The fused features retain the acoustic continuity information of the first audio segment and the previous audio segment, and highlight the semantic logical connection, significantly improving the accuracy of splicing judgment, especially suitable for semantic integrity judgment of non-natural pauses in vehicle scenarios.

[0015] In an optional implementation, the judgment module further includes a splicing judgment unit; the splicing judgment unit is used for: Based on the fusion features and the preset attention algorithm, the splicing prediction probability and the non-splicing prediction probability are obtained respectively. Based on the splicing prediction probability and the non-splicing prediction probability, determine whether to splice the first audio segment and the previous audio segment.

[0016] The embodiments of this application obtain the splicing prediction probability and the non-splicing prediction probability respectively through a preset attention algorithm. While ensuring the accuracy of splicing judgment, the computational complexity is reduced to the minimum. With the parallel computing design, the synchronous completion of text acquisition and feature fusion / splicing judgment is ensured, and the response speed is further optimized.

[0017] In conjunction with the first aspect, in one optional implementation, the voice processing system further includes: a text adjustment module and a control module; The text adjustment module is used for: After splicing, the spliced ​​text is obtained; the splicing confidence score of the spliced ​​text is determined, the splicing confidence score includes acoustic coherence score and semantic logic score; When the splicing confidence score is less than the confidence threshold, the spliced ​​text is modified to obtain the modified spliced ​​text; The control module is used for: When the splicing confidence score is greater than or equal to the confidence threshold, a vehicle control command is generated based on the spliced ​​text; or, Based on the modified concatenated text, vehicle control commands are generated.

[0018] The splicing confidence score in this application includes acoustic coherence score and semantic logic score, which quantifies acoustic coherence and semantic logic. When the splicing confidence score is low, the spliced ​​text is modified to finally generate vehicle control commands. This not only ensures the reliability of the spliced ​​text through the confidence mechanism, but also repairs potential semantic breaks through dynamic adjustment, ensuring the accuracy of subsequent vehicle control commands and improving the robustness of intelligent cockpit voice interaction.

[0019] In conjunction with the first aspect, in one alternative implementation, the following two steps are performed simultaneously: The character recognition is performed based on the first feature information to obtain the first character recognition result; The first feature information is fused with the second feature information of the previous audio segment in the audio segment sequence to obtain the fused feature; Alternatively, the following two steps can be performed simultaneously: The character recognition is performed based on the first feature information to obtain the first character recognition result; Based on the fusion features, it is determined whether the first audio segment needs to be spliced ​​with the previous audio segment.

[0020] Secondly, embodiments of this application provide a voice processing method, the method comprising: The first audio segment in the audio segment sequence is subjected to feature recognition to obtain the first feature information; Upon obtaining the first feature information, the text conversion mechanism and splicing judgment mechanism are triggered. According to the text conversion mechanism, character recognition is performed based on the first feature information to obtain the first character recognition result; According to the splicing judgment mechanism, the first feature information is fused with the second feature information of the previous audio segment of the first audio segment in the audio segment sequence to obtain a fused feature; based on the fused feature, it is determined whether the first audio segment and the previous audio segment need to be spliced. When the judgment result is yes, the first character recognition result and the second character recognition result corresponding to the previous audio segment are obtained and concatenated.

[0021] Thirdly, embodiments of this application provide a vehicle, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the above-mentioned second aspect or its corresponding voice processing method.

[0022] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which are used to cause a computer to perform the second aspect or the corresponding speech processing method described above.

[0023] Fifthly, this application provides a computer program product, including computer instructions for causing a computer to execute the speech processing method described in the second aspect above or its corresponding method. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of a conventional speech processing flow according to an embodiment of this application; Figure 2 This is a schematic diagram of the structure of the voice processing system according to an embodiment of this application; Figure 3 This is a schematic diagram of the voice processing flow according to an embodiment of this application; Figure 4 This is a schematic flowchart of the speech processing method according to an embodiment of this application; Figure 5 This is a schematic diagram of the hardware structure of the vehicle according to an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] Traditional methods determine whether to splice two segments of speech by setting a silence duration threshold. For example, if a user says, "Please play... (pause for 3 seconds) A (song name)," and the silence duration of "Please play" and "A (song name)" reaches the silence duration threshold, the traditional method directly splits the complete semantics into two independent segments: "Please play" and "A (song name)." This causes Natural Language Understanding (NLU) systems to be unable to accurately interpret user requests due to the lack of key information (such as operation instructions and operation objects), thus triggering invalid responses or failing to execute requests.

[0028] To alleviate the above problems, related technologies propose two optimization schemes. The first optimization scheme is to extend the silence duration threshold to reduce erroneous segmentation caused by unnatural pauses. However, a longer silence duration threshold will cause a new contradiction: for truly long speech inputs, segmentation can only be completed after the threshold duration has expired, resulting in increased speech processing latency.

[0029] The second optimization approach involves training a new, independent model to determine the acoustic feature correlation between adjacent speech segments, or simultaneously combining the correlation between acoustic and semantic features, to decide whether to splice them together. For example... Figure 1 As shown, the input to the new model is acoustic features extracted from audio segments, or acoustic features combined with semantic features. This approach, by introducing an additional semantic concatenation system to specifically determine whether the current segment request should be concatenated with the previous segment, effectively avoids the problem of increased speech processing latency caused by the inability to segment long speech segments in a timely manner due to extending the silence duration threshold. However, the drawback is that the additionally constructed semantic concatenation system requires separate feature extraction, computation, and inference, which significantly increases the overall computational load of the system and increases response latency. In real-time demanding in-vehicle voice interaction scenarios, this latency can severely disrupt the user's conversation rhythm and significantly degrade the user experience.

[0030] Therefore, embodiments of this application provide a voice processing system to solve the above-mentioned problems. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0031] Figure 2 This is a schematic diagram of the structure of the voice processing system according to an embodiment of this application.

[0032] This embodiment provides a voice processing system, such as Figure 2 As shown, the speech processing system includes: The recognition module is used to perform feature recognition on the first audio segment in the audio segment sequence to obtain first feature information. Based on the first feature information, text recognition is performed to obtain the first text recognition result.

[0033] The judgment module is used to fuse the first feature information with the second feature information of the previous audio segment in the audio segment sequence to obtain the fused feature. Based on the fused feature, it determines whether the first audio segment needs to be concatenated with the previous audio segment.

[0034] The splicing module is used to obtain the first character recognition result and the second character recognition result corresponding to the previous audio segment when the judgment result is yes, and then splice them together.

[0035] In this embodiment of the application, the following two steps are performed simultaneously: Based on the first feature information, character recognition is performed to obtain the first character recognition result.

[0036] The first feature information is fused with the second feature information of the previous audio segment in the audio segment sequence to obtain the fused feature.

[0037] Alternatively, the following two steps can be performed simultaneously: Based on the first feature information, character recognition is performed to obtain the first character recognition result.

[0038] Based on the fusion characteristics, it is determined whether the first audio segment needs to be spliced ​​with the previous audio segment.

[0039] In this embodiment, the feature information may include acoustic features and semantic features. The audio segment is obtained through speech activity detection technology. The speech activity detection system continuously monitors the input speech stream, and when the detected silence duration exceeds a preset threshold, it segments the currently detected speech portion into an audio segment. The preset threshold can be flexibly adjusted according to the needs of the in-vehicle scenario to balance segmentation accuracy and response speed, enabling timely segmentation of long speech without needing to additionally extend the silence duration threshold to avoid recognizing two speech segments with the same requirement as two independent audio fragments for separate processing.

[0040] The recognition module retrieves these segmented audio segments sequentially from the speech activity detection system, according to the temporal order of the speech stream. The recognition module can be an Automatic Speech Recognition (ASR) system.

[0041] The speech processing system also includes a storage module for storing the first feature information of the first audio segment and the first text recognition result corresponding to the first audio segment. The storage module can be stored in a designated storage area of ​​the system, such as a memory cache or a specific database table.

[0042] The second feature information of the previous audio segment of the first audio segment is also stored in a designated storage area of ​​the system, such as a memory cache or a specific database table, so that the subsequent judgment module can quickly retrieve it when fusing the first feature information with the second feature information of the previous audio segment in the audio segment sequence. The second character recognition result corresponding to the previous audio segment of the first audio segment is also stored in a designated storage area of ​​the system, such as a memory cache or a specific database table, so that the subsequent splicing module can quickly retrieve it when splicing the first character recognition result with the second character recognition result corresponding to the previous audio segment.

[0043] In this embodiment, the splicing module can obtain fusion features based on the first feature information of the first audio segment and the second feature information of the previous audio segment. Based on the fusion features, it determines whether to splice the first audio segment and the previous audio segment, obtaining a judgment result, which includes yes (splicing) or no (not splicing).

[0044] In this embodiment, when the splicing module splices the first text recognition result with the second text recognition result corresponding to the previous audio segment, it splices the first text recognition result and the second text recognition result in chronological order to obtain a complete spliced ​​text, so that the user's intent can be accurately interpreted based on the complete spliced ​​text, and then corresponding vehicle control commands and other operation commands can be generated.

[0045] This application embodiment obtains a fusion feature by fusing the second feature information of the first audio segment and the previous audio segment, and improves the accuracy of splicing judgment by using the fusion feature to determine whether splicing should be performed.

[0046] In the process of obtaining fused features by fusing the second feature information of the first audio segment and the previous audio segment, or in the process of determining whether to splice based on the fused features, the first text recognition result corresponding to the first audio segment is simultaneously acquired. The text acquisition of the audio segment and the feature fusion / sponging judgment are executed in parallel, eliminating the need to wait for the fused feature / judgment result before starting text acquisition, or to perform feature fusion / sponging judgment only after completing text acquisition. This parallel design enables the synchronous generation of fused feature / sponging results and text output, effectively improving the time consumption caused by serial processing, significantly enhancing speech splicing efficiency, and effectively reducing semantic response latency, thus meeting the high requirements of real-time interaction in smart cockpits. This application can acquire the feature information of the audio segment during the speech-to-text process, and determine whether splicing is needed based on the feature information of the audio segment acquired during speech-to-text. This enables the synchronous generation of fused feature / sponging results and text output, eliminating the need to separately acquire the feature information of the audio segment, effectively improving speech splicing efficiency and reducing speech response latency.

[0047] When determining the splicing, the pre-stored second character recognition result is integrated with the first character recognition result to ensure that the semantics corresponding to the two segmented speech segments can be completely connected, avoiding command execution failure due to text breakage and improving the reliability of voice interaction.

[0048] The fusion judgment is based on the feature information of existing audio, without the need to introduce additional models or calculation modules to determine the feature information. This improves the efficiency of speech splicing, reduces speech response latency, and avoids excessive consumption of system resources.

[0049] In one alternative implementation, the first feature information includes acoustic and semantic features of the first audio segment, and the recognition module includes an encoder and a decoder.

[0050] The encoder receives a first audio segment and performs feature recognition on it to obtain its acoustic features. The acoustic features of the first audio segment are the feature sequence output by the encoder's top hidden layer. These acoustic features preserve complete acoustic details and temporal continuity information.

[0051] The decoder receives the acoustic features of the first audio segment and derives its semantic features based on these features. The semantic features of the first audio segment are the feature sequence output by the top hidden layer of the decoder.

[0052] The judgment module of this application directly reuses the hidden layer output of the encoder and decoder of the recognition module (automatic speech recognition system) as acoustic features and semantic features. There is no need to design an additional feature extraction module to extract feature information. This realizes feature reuse based on the recognition module, which reduces the computational overhead of feature extraction and ensures the correlation between acoustic features and semantic features.

[0053] In one alternative implementation, the recognition module further includes a text conversion unit.

[0054] The text conversion unit is used to perform character recognition based on the acoustic and semantic features of the first audio segment to obtain the first character recognition result.

[0055] In this embodiment, the text conversion unit of the recognition module performs text recognition based on the acoustic features of the first audio segment obtained by the encoder and the semantic features of the first audio segment obtained by the decoder, and obtains the first text recognition result. The judgment module can reuse the first feature information of the first audio segment without affecting the process of the recognition module performing text recognition based on the first feature information of the first audio segment and obtaining the first text recognition result. By executing in parallel, the delay caused by serial processing is avoided, and the immediacy of voice interaction is significantly improved.

[0056] In one optional implementation, the decision module includes a fusion unit. Specifically, the fusion unit is based on a cross-attention network to fuse acoustic and semantic features.

[0057] Fusion unit, used for: The acoustic features of the first audio segment are spliced ​​together with the acoustic features of the previous audio segment to obtain the spliced ​​acoustic features.

[0058] The semantic features of the first audio segment and the semantic features of the previous audio segment are concatenated to obtain the concatenated semantic features.

[0059] The key features of the preset categories are selected from the concatenated semantic features to obtain the query matrix.

[0060] Based on the query matrix, extract at least two key acoustic features that are semantically related to the spliced ​​semantic features from the spliced ​​acoustic features.

[0061] At least two types of key acoustic features are fused to obtain fused features.

[0062] In this embodiment, if the previous judgment result of whether splicing was performed was negative, the fusion unit is configured to: splice the acoustic features of the first audio segment and the acoustic features of the previous audio segment to obtain spliced ​​acoustic features; splice the semantic features of the first audio segment and the semantic features of the previous audio segment to obtain spliced ​​semantic features; filter out key features of a preset category from the spliced ​​semantic features to obtain a query matrix; extract at least two types of key acoustic features that are semantically related to the spliced ​​semantic features from the spliced ​​acoustic features according to the query matrix; and fuse the at least two types of key acoustic features to obtain fused features. The corresponding splicing module is specifically configured to, when the judgment result is positive and the previous judgment result of whether splicing was performed was negative, obtain the first character recognition result and the second character recognition result corresponding to the previous audio segment, and splice them together.

[0063] For example, audio segment A, audio segment B, and audio segment C. If the concatenation result of audio segment A and audio segment B is negative, then audio segment C is the first audio segment, and audio segment B is the audio segment preceding the first audio segment (audio segment C).

[0064] The previous audio segment (audio segment C) is the first independent audio segment that is immediately adjacent to and generated before the first audio segment (audio segment B). The second feature information of the previous audio segment (audio segment C) and the second character recognition result corresponding to the previous audio segment (audio segment C) are also obtained through the recognition module.

[0065] In this embodiment, if the result of the previous determination of whether splicing was performed is yes, then the previous audio segment of the first audio segment is the audio segment after the previous splicing. For example, audio segment A, audio segment B, and audio segment C. If the splicing result of audio segment A and audio segment B is yes, then audio segment C is the first audio segment, and audio segment A + audio segment B is the previous audio segment of the first audio segment (audio segment C).

[0066] If the previous judgment result for whether to perform splicing was yes, the fusion unit is further configured to: splice the acoustic features of the first audio segment and the first spliced ​​acoustic features (the spliced ​​acoustic features obtained by the previous fusion unit) to obtain the second spliced ​​acoustic features; splice the semantic features of the first audio segment and the first spliced ​​semantic features (the spliced ​​semantic features obtained by the previous fusion unit) to obtain the second spliced ​​semantic features; select key features of a preset category from the second spliced ​​semantic features to obtain a query matrix; extract at least two key acoustic features that are semantically related to the spliced ​​semantic features from the second spliced ​​acoustic features according to the query matrix; and fuse the at least two key acoustic features to obtain fused features. The first spliced ​​acoustic feature is the spliced ​​acoustic feature obtained by the previous fusion unit. The first spliced ​​semantic feature is the spliced ​​semantic feature obtained by the previous fusion unit. The corresponding splicing module is further configured to, when the judgment result is yes and the previous judgment result is yes, obtain the first character recognition result and the first spliced ​​text (the spliced ​​text obtained by the previous splicing module) and splice them together.

[0067] The storage module is also used to store the splicing acoustic features and splicing semantic features obtained by the fusion unit when the judgment result is yes, and to store the splicing text obtained by the splicing module. It can be stored in a designated storage area of ​​the system, such as memory cache or a specific database table.

[0068] In this embodiment, the acoustic features of the first audio segment and the acoustic features of the previous audio segment are concatenated to obtain concatenated acoustic features. Specifically, the acoustic features of the first audio segment (length m1) and the acoustic features of the previous audio segment (length m2) are concatenated in the temporal dimension to obtain concatenated acoustic features with a total length of m1 + m2, thus fully preserving the acoustic continuity information and temporal correlation of the two audio segments.

[0069] In this embodiment, the semantic features of the first audio segment and the semantic features of the previous audio segment are concatenated to obtain concatenated semantic features. Specifically, the semantic features of the first audio segment (length n1) and the semantic features of the previous audio segment (length n2) are concatenated along the text dimension to obtain concatenated semantic features with a total length of n1+n2, thus preserving the semantic relationship between the text context of the two audio segments.

[0070] In this embodiment, key features of preset categories are selected from the concatenated semantic features to obtain a query matrix. Specifically, based on the semantic concatenation requirements of the in-vehicle scenario, the preset categories typically include: intent keyword features, such as features corresponding to operation intent words like play, navigation, and open; core object features, such as features corresponding to operation objects like song names and destinations; and logical association features, such as features corresponding to logical words connecting intents and objects like go and arrive. The sequence length of the query matrix is ​​consistent with the sequence length of the concatenated semantic features. Its core function is to serve as a retrieval instruction for the cross-attention network, clarifying which semantically strongly related acoustic features need to be selected from the concatenated acoustic features, thus avoiding redundant calculations caused by indiscriminately processing acoustic features.

[0071] In this embodiment, based on the query matrix, at least two types of key acoustic features semantically related to the concatenated semantic features are extracted from the concatenated acoustic features. Specifically, the cross-attention network predefines a key weight matrix, performs matrix multiplication on the concatenated acoustic features and the key weight matrix to obtain the key matrix corresponding to the concatenated acoustic features. The dot product similarity between the query matrix and the key matrix corresponding to the concatenated acoustic features is calculated to obtain a similarity matrix. Each element in the similarity matrix is ​​divided by a scaling factor (the square root of the key matrix dimension). Dividing by the scaling factor scales the similarity values ​​to a reasonable range (usually close to a normal distribution with a mean of 0 and a variance of 1), avoiding extreme values ​​and smoothing the similarity distribution. The scaled similarity matrix is ​​processed using the softmax function to obtain the attention weight matrix. The higher the weight value, the stronger the correlation between the corresponding acoustic feature and the semantics. Through the filtering of the attention weight matrix, only acoustic features with high weights are retained as key acoustic features. For example, acoustic features with weight values ​​greater than a preset weight value can be used as key acoustic features.

[0072] In this embodiment, the cross-attention network predefines a key weight matrix. The concatenated acoustic features are multiplied by this key weight matrix to obtain the key matrix corresponding to the concatenated acoustic features. The dot product similarity between the query matrix and the key matrix corresponding to the concatenated acoustic features is calculated to obtain a similarity matrix. Each element in the similarity matrix is ​​divided by a scaling factor (the square root of the key matrix dimension). Dividing by the scaling factor scales the similarity values ​​to a reasonable range (typically close to a normal distribution with a mean of 0 and a variance of 1), avoiding extreme values ​​and smoothing the similarity distribution.

[0073] In this embodiment, at least two types of key acoustic features are fused to obtain fused features. Specifically, in the cross-attention network, the selected key acoustic features correspond to a value matrix, and the attention weight matrix and the value matrix are weighted and summed to obtain the fused features. The sequence length of the fused features is consistent with the sequence length of the query matrix. The cross-attention network retains only key acoustic features that are strongly correlated with semantics and eliminates redundant acoustic features. Therefore, the sequence length of the fused features is shorter than the sequence length of the original concatenated acoustic features, achieving effective compression of the sequence length.

[0074] This application embodiment extracts key acoustic features based on the query matrix generated by splicing semantic features, realizing the deep fusion of acoustic features and semantic features. The fused features retain the acoustic continuity information of the first audio segment and the previous audio segment, and highlight the semantic logical connection, significantly improving the accuracy of splicing judgment, especially suitable for semantic integrity judgment of non-natural pauses in vehicle scenarios.

[0075] In an optional implementation, the determination module further includes a splicing determination unit. The splicing determination unit is used for: Based on the fusion features and the preset attention algorithm, the splicing prediction probability and the non-splicing prediction probability are obtained respectively.

[0076] Based on the splicing prediction probability and the non-splicing prediction probability, determine whether to splice the first audio segment and the previous audio segment.

[0077] In this embodiment, the preset attention algorithm is a conventional attention-based classification algorithm in the art, such as an algorithm combining attention pooling algorithm with shallow multilayer perceptron classification algorithm (such as two-layer multilayer perceptron classification algorithm). Shallow multilayer perceptron classification algorithm, such as two-layer multilayer perceptron classification algorithm, has very few parameters and low computational cost. The time consumption of the two processes of audio segment text acquisition and feature fusion / concatenation judgment is basically the same. Therefore, the parallel execution of the two processes of audio segment text acquisition and feature fusion / concatenation judgment can be realized, improving the overall efficiency of speech processing.

[0078] In this embodiment, obtaining the concatenation prediction probability and the non-concatenation prediction probability based on fused features and a preset attention algorithm includes: performing attention pooling on the fused features to obtain a global feature vector; performing classification mapping on the global feature vector through two fully connected layers to obtain a binary classification feature vector; and normalizing the binary classification feature vector using an activation function to obtain the concatenation prediction probability and the non-concatenation prediction probability.

[0079] In this embodiment, attention pooling refers to an attention weighting operation that filters key information and compresses sequences in the fused features. Specifically, it includes: pre-setting an attention weight vector; performing a dot product operation between this attention weight vector and the feature vector at each position in the fused features to obtain an original attention score reflecting the importance of each position to the splicing judgment; normalizing all original attention scores using the softmax function to obtain a probability distribution with a sum of 1 (higher weights indicate more critical fused features at the corresponding position); and element-wise summing the feature vectors at each position of the fused features with their corresponding normalized weights to finally obtain a feature vector with a sequence length compressed to 1 and a dimension consistent with the attention weight vector.

[0080] The global feature vector refers to the output of the attention pooling process.

[0081] The two fully connected layers consist of a first fully connected layer and a second fully connected layer.

[0082] The classification mapping of global feature vectors using two fully connected layers refers to: The first fully connected layer performs a dimensionality transformation on the global feature vector to obtain an intermediate feature vector. The second fully connected layer performs a binary classification space mapping on the intermediate feature vector to obtain a binary classification feature vector.

[0083] The first fully connected layer performs a dimensionality transformation on the global feature vector to obtain an intermediate feature vector. The dimensions of the intermediate feature vector, the weight matrix W1, and the bias vector b1 of the first fully connected layer are preset. The dimension of the global feature vector is d, and the dimension of the intermediate feature vector is d1. Typically, d1 is close to or twice the size of d; for example, when d=256, d1=256 or d1=512 is set to avoid excessively high dimensionality leading to increased computational cost.

[0084] The input global feature vector is batch-multiplied with the weight matrix W1 to obtain the linearly transformed feature matrix X, thus mapping the global feature vector from the original dimension d to the preset intermediate dimension d1. A bias vector b1 is introduced, and the bias vector b1 is added element-wise to the feature matrix X to obtain X_b = X + b1.

[0085] Choose a lightweight activation function, such as the ReLU activation function. The ReLU function performs non-linear processing on X_b, setting all negative features to 0 and retaining positive features, ultimately obtaining the intermediate feature vector X_mid.

[0086] The second fully connected layer performs a binary classification space mapping on the intermediate feature vectors to obtain binary classification feature vectors.

[0087] The input to the second fully connected layer is the intermediate feature vector X_mid, with the same dimension as the output dimension of the first layer.

[0088] The output dimension is fixed at 2 (corresponding to the concatenated and unconcatenated categories respectively).

[0089] The weight matrix W2 and bias vector b2 of the second fully connected layer are preset.

[0090] Batch matrix multiplication is performed between the intermediate feature vector X_mid and the weight matrix W2 to obtain the linearly transformed classification score matrix X_c. This directly maps the intermediate features from the d1 dimension to the 2-dimensional classification space, so that each dimension corresponds to the original score of a class.

[0091] A bias vector b2 is introduced, and this bias vector b2 is added element-wise to X_c to obtain X_cl = X_c + b2. The introduction of the bias term can adjust the base scores of the two categories, avoiding excessive bias towards one category due to weight bias.

[0092] Unlike the first layer, the second fully connected layer does not introduce non-linear activation functions such as ReLU (to avoid destroying the original differences in classification scores), and directly outputs X_cl as the binary classification feature vector. Each element in the binary classification feature vector represents the original discrimination score of the corresponding class. For example, in [1.9, 0.4], 1.9 corresponds to the original score of "concatenation", and 0.4 corresponds to the original score of "no concatenation". Although it does not have probabilistic meaning, it can reflect the model's initial tendency towards the two classes.

[0093] The embodiments of this application obtain the splicing prediction probability and the non-splicing prediction probability respectively through a preset attention algorithm. While ensuring the accuracy of splicing judgment, the computational complexity is reduced to the minimum. With the parallel computing design, the synchronous completion of text acquisition and feature fusion / splicing judgment is ensured, and the response speed is further optimized.

[0094] This application embodiment extracts key acoustic features based on a query matrix, achieving deep fusion of acoustic and semantic features. The fused features retain the acoustic continuity information of the two audio segments while highlighting the semantic logical connection, significantly improving the accuracy of splicing judgment, especially suitable for semantic integrity judgment of non-natural pauses in vehicle scenarios.

[0095] The voice processing flow of this application embodiment is as follows: Figure 3 As shown, speech activity detection technology continuously monitors the input speech stream and segments the continuous speech stream into audio segments. When the silence duration exceeds a preset threshold, the currently detected speech portion is segmented into an audio segment. These segmented audio segments are the basic input for all subsequent processing.

[0096] The recognition module (automatic speech recognition system) comprises two core parts: an encoder and a decoder, as well as a text conversion unit. The encoder receives a first audio segment and performs feature recognition on it to obtain its acoustic features. The decoder receives the acoustic features of the first audio segment and derives its semantic features based on these features. The text conversion unit performs character recognition based on the acoustic and semantic features of the first audio segment to obtain the first character recognition result.

[0097] The judgment module receives multiple types of input: acoustic and semantic features of the previous audio segment of the first audio segment, and acoustic and semantic features of the first audio segment, to determine whether to splice audio segments.

[0098] During the process of determining whether to splice, the decoding of the first audio segment is triggered simultaneously, generating the first text recognition result. This works in conjunction with the text conversion unit function of the recognition module to ensure that text generation and splicing determination are synchronized.

[0099] When the judgment module determines that concatenation is successful, it acquires the first character recognition result and the second character recognition result corresponding to the previous audio segment, and concatenates them to obtain the concatenated text. This concatenated text is then passed to the natural language understanding system. If the judgment module determines that concatenation is not necessary, the concatenation module directly passes the first character recognition result. The natural language understanding system will further analyze the semantics of this text, such as recognizing user commands like "turn on the air conditioner," thereby triggering subsequent air conditioner control actions.

[0100] In one alternative implementation, the voice processing system further includes a text adjustment module and a control module.

[0101] The text adjustment module is used for: After splicing, obtain the spliced ​​text. Determine the splicing confidence score of the spliced ​​text, which includes acoustic coherence score and semantic logicality score.

[0102] When the confidence score of the concatenated text is less than the confidence threshold, the concatenated text is modified to obtain the modified concatenated text.

[0103] The control module is used for: When the concatenation confidence score is greater than or equal to the confidence threshold, vehicle control commands are generated based on the concatenated text. Alternatively, vehicle control commands are generated based on the modified concatenated text.

[0104] The concatenation confidence score of the concatenated text is determined. Specifically, the acoustic coherence score can be calculated using the spectral smoothness and prosodic consistency parameters at the junction of the two audio segments in the fusion features. These parameters can be the fundamental frequency variation amplitude and the inter-frame energy difference. The semantic logicality score can be obtained by calculating the contextual semantic similarity of the concatenated text, which can be cosine similarity. The concatenation confidence score is obtained by weighted summation of the acoustic coherence score and the semantic logicality score using preset weights.

[0105] When the concatenation confidence score is less than the confidence threshold, the concatenated text is modified to obtain the modified concatenated text. Specifically, the need to modify the concatenated text can be determined based on the concatenation confidence score. A confidence threshold adapted to the needs of in-vehicle voice interaction is preset, such as 0.6. The confidence threshold can be dynamically adjusted according to different vehicle control scenarios. The calculated concatenation confidence score is compared with this confidence threshold. If the concatenation confidence score is greater than or equal to the confidence threshold, it is determined that no change to the concatenated text is needed. If the concatenation confidence score is less than the confidence threshold, it is determined that the concatenation text needs to be modified.

[0106] As an example, a specific method for modifying concatenated text could be to identify the key connection points corresponding to semantic breaks or acoustic inconsistencies in the concatenated text. For instance, for the concatenated text "navigate to" and "a certain park," the key connection point is determined to be between "go" and "a certain park." Then, a semantic completion knowledge base adapted for in-vehicle scenarios is invoked. This knowledge base pre-stores common semantic break repair rules and high-frequency connection components under different vehicle control scenarios (such as navigation and multimedia control). For navigation scenarios, "destination" is added to the aforementioned connection point to form "navigate to destination a certain park," strengthening the association between the instruction and the target. For multimedia control scenarios, such as the concatenated text "please play" and "A" (a song name), "song" is added to form "please play song A," completing the operation object information. After adjustment, the concatenation confidence score of the adjusted concatenated text is recalculated. If the concatenation confidence score is greater than or equal to the confidence threshold, it is used as the final concatenated text. If the concatenation confidence score is still less than the confidence threshold, a pre-trained language model for in-vehicle scenarios, such as a lightweight BERT model, is further invoked to optimize the sentence structure of the concatenated text, such as adjusting word order and supplementing grammatical components, until the concatenation confidence score is greater than or equal to the confidence threshold, resulting in an adjusted concatenated text to ensure that the subsequently generated vehicle control commands can accurately match the user's intent.

[0107] As an example, a specific way to determine the key connection points corresponding to semantic breaks or acoustic incoherence in spliced ​​text could be to extract local acoustic parameters at the connection points between two audio segments from the spliced ​​acoustic features. These local acoustic parameters could include inter-frame spectral similarity, fundamental frequency change rate, and energy mutation amplitude.

[0108] As an example, the inter-frame spectral similarity can be obtained by calculating the Mel-frequency cosine distance between audio frames 10-20ms before and after the junction. As an example, the fundamental frequency change rate can be obtained by calculating the ratio of the difference in fundamental frequency values ​​before and after the junction to the average fundamental frequency of the preceding segment. As an example, the energy change amplitude can be obtained by calculating the difference in energy values ​​before and after the junction. When local acoustic parameters exceed a preset acoustic threshold and meet preset conditions, that location (the junction of two audio segments) is marked as a critical junction location. As an example, the preset conditions could be a spectral similarity < 0.6 and a fundamental frequency change rate > 30%.

[0109] The splicing confidence score in this application includes acoustic coherence score and semantic logic score, which quantifies acoustic coherence and semantic logic. When the splicing confidence score is low, the spliced ​​text is modified to finally generate vehicle control commands. This not only ensures the reliability of the spliced ​​text through the confidence mechanism, but also repairs potential semantic breaks through dynamic adjustment, ensuring the accuracy of subsequent vehicle control commands and improving the robustness of intelligent cockpit voice interaction.

[0110] In this embodiment, the control module generates vehicle control commands based on the concatenated text. Alternatively, it generates vehicle control commands based on the modified concatenated text. For example, if the concatenated / modified concatenated text is "Please adjust the air conditioning temperature to 23 degrees," the corresponding vehicle control command is {"Control Module": "Air Conditioning System", "Command Type": "Temperature Adjustment", "Parameters": {"Target Temperature": 23", "Temperature Unit": "°C", "Adjustment Mode": "Precise Setting"}}. If the concatenated / modified concatenated text is "Open the driver's side window to halfway," the corresponding vehicle control command is {"Control Module": "Window System", "Command Type": "Window Control", "Parameters": {"Window Position": "Driver's Side", "Target Opening Degree": 50", "Action Type": "Open"}}.

[0111] In this embodiment, the splicing module is further configured to input the first character recognition result into the text adjustment module when the judgment result is negative. The text adjustment module is further configured to: determine the splicing confidence score of the first character recognition result, the splicing confidence score including acoustic coherence score and semantic logic score. When the splicing confidence score is less than a confidence threshold, the first character recognition result is modified to obtain a modified first character recognition result. The control module is further configured to: generate vehicle control commands based on the first character recognition result when the splicing confidence score is greater than or equal to the confidence threshold. Or, generate vehicle control commands based on the modified first character recognition result.

[0112] In this embodiment, the voice processing system is presented in the form of functional units. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0113] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0114] This application provides an embodiment of a voice processing method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0115] Figure 4 This is a flowchart illustrating a speech processing method according to an embodiment of this application. The method is applied to a speech processing system, which includes a recognition module, a judgment module, and a splicing module.

[0116] like Figure 4 As shown, the process includes the following steps: S101: Perform feature recognition on the first audio segment in the audio segment sequence to obtain the first feature information.

[0117] S102: Upon obtaining the first feature information, the text conversion mechanism and splicing judgment mechanism are triggered.

[0118] S103: Based on the text conversion mechanism, perform character recognition based on the first feature information to obtain the first character recognition result.

[0119] The identification module is used to perform steps S101, S102 and S103.

[0120] S104: According to the splicing judgment mechanism, the first feature information is fused with the second feature information of the previous audio segment in the audio segment sequence to obtain the fused feature. Based on the fused feature, it is determined whether the first audio segment needs to be spliced ​​with the previous audio segment.

[0121] The decision module is used to execute step S104.

[0122] S105: When the judgment result is yes, obtain the first character recognition result and the second character recognition result corresponding to the previous audio segment, and concatenate them.

[0123] The splicing module is used to execute the steps in S105.

[0124] In one alternative implementation, the first feature information includes acoustic and semantic features of the first audio segment, and the recognition module includes an encoder and a decoder.

[0125] S101 performs feature recognition on the first audio segment in the audio segment sequence, obtaining the first feature information including: The acoustic features of the first audio segment are obtained by feature recognition of the first audio segment through the encoder.

[0126] The acoustic features of the first audio segment are received by the decoder, and the semantic features of the first audio segment are obtained based on the acoustic features of the first audio segment.

[0127] In one alternative implementation, the recognition module further includes a text conversion unit.

[0128] S103, based on the text conversion mechanism, performs character recognition based on the first feature information to obtain the first character recognition result, including: The text conversion unit performs character recognition based on the acoustic and semantic features of the first audio segment to obtain the first character recognition result.

[0129] In one alternative implementation, the determination module includes a fusion unit.

[0130] In S104, based on the splicing judgment mechanism, the first feature information is fused with the second feature information of the previous audio segment in the audio segment sequence to obtain the fused feature, including: The acoustic features of the first audio segment and the previous audio segment are spliced ​​together by the fusion unit to obtain the spliced ​​acoustic features.

[0131] The semantic features of the first audio segment and the semantic features of the previous audio segment are concatenated to obtain the concatenated semantic features.

[0132] The key features of the preset categories are selected from the concatenated semantic features to obtain the query matrix.

[0133] Based on the query matrix, extract at least two key acoustic features that are semantically related to the spliced ​​semantic features from the spliced ​​acoustic features.

[0134] At least two types of key acoustic features are fused to obtain fused features.

[0135] In one optional implementation, the judgment module further includes a splicing judgment unit.

[0136] S104, based on fusion features, determines whether the first audio segment needs to be spliced ​​with the previous audio segment, including: Based on fusion features and a preset attention algorithm, the splicing judgment unit obtains the splicing prediction probability and the non-splicing prediction probability, respectively.

[0137] Based on the splicing prediction probability and the non-splicing prediction probability, determine whether to splice the first audio segment and the previous audio segment.

[0138] In one alternative implementation, the voice processing system further includes a text adjustment module and a control module.

[0139] When the judgment result is yes in S105, the method further includes obtaining the first character recognition result and the second character recognition result corresponding to the previous audio segment, and concatenating them. After splicing, the text adjustment module obtains the spliced ​​text. The splicing confidence score is then determined, which includes acoustic coherence and semantic logic scores.

[0140] When the confidence score of the concatenated text is less than the confidence threshold, the concatenated text is modified to obtain the modified concatenated text.

[0141] The control module generates vehicle control commands based on the concatenated text when the concatenated confidence score is greater than or equal to the confidence threshold. Alternatively, it generates vehicle control commands based on the modified concatenated text.

[0142] In one alternative implementation, the following two steps are performed simultaneously: Based on the first feature information, character recognition is performed to obtain the first character recognition result.

[0143] The first feature information is fused with the second feature information of the previous audio segment in the audio segment sequence to obtain the fused feature.

[0144] Alternatively, the following two steps can be performed simultaneously: Based on the first feature information, character recognition is performed to obtain the first character recognition result.

[0145] Based on the fusion characteristics, it is determined whether the first audio segment needs to be spliced ​​with the previous audio segment.

[0146] This application also provides a vehicle having the above-described features. Figure 4 The speech processing method shown.

[0147] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a vehicle provided in an optional embodiment of this application, such as... Figure 5 As shown, the vehicle includes one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the vehicle, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple vehicles can be connected, with each device providing some of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). Figure 5 Take a processor 10 as an example.

[0148] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0149] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0150] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store data created based on vehicle usage as displayed on a mini-program landing page. Furthermore, the memory 20 may include high-speed random access memory and non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the vehicle via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0151] Memory 20 may include volatile memory, such as random access memory. Memory may also include non-volatile memory, such as flash memory, hard disk, or solid-state drive. Memory 20 may also include combinations of the above types of memory.

[0152] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the vehicle, such as a touchscreen, keypad, mouse, trackpad, touchpad, indicator, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0153] The vehicle also includes a communication interface 30 for communicating with other devices or communication networks.

[0154] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc. Further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0155] A portion of this application can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to this application through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0156] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A voice processing system, characterized in that, The system includes: The recognition module is used to perform feature recognition on the first audio segment in the audio segment sequence to obtain first feature information; and to perform character recognition based on the first feature information to obtain a first character recognition result. The judgment module is used to fuse the first feature information with the second feature information of the previous audio segment of the first audio segment in the audio segment sequence to obtain a fused feature; based on the fused feature, it is determined whether the first audio segment and the previous audio segment need to be spliced ​​together. The splicing module is used to obtain the first text recognition result and the second text recognition result corresponding to the previous audio segment when the judgment result is yes, and then splice them together.

2. The system according to claim 1, characterized in that, The first feature information includes the acoustic features and semantic features of the first audio segment, and the recognition module includes an encoder and a decoder; The encoder is used to receive the first audio segment and perform feature recognition on the first audio segment to obtain the acoustic features of the first audio segment. The decoder is used to receive the acoustic features of the first audio segment and obtain the semantic features of the first audio segment based on the acoustic features of the first audio segment.

3. The system according to claim 2, characterized in that, The recognition module further includes: a text conversion unit; The text conversion unit is used to perform character recognition based on the acoustic and semantic features of the first audio segment to obtain the first character recognition result.

4. The system according to claim 2, characterized in that, The judgment module includes a fusion unit; The fusion unit is used for: The acoustic features of the first audio segment and the acoustic features of the previous audio segment are spliced ​​together to obtain spliced ​​acoustic features. The semantic features of the first audio segment and the semantic features of the previous audio segment are concatenated to obtain the concatenated semantic features. From the concatenated semantic features, key features of preset categories are selected to obtain a query matrix; Based on the query matrix, extract at least two types of key acoustic features that are semantically related to the spliced ​​semantic features from the spliced ​​acoustic features; The at least two types of key acoustic features are fused to obtain the fused features.

5. The system according to claim 4, characterized in that, The judgment module further includes a splicing judgment unit; the splicing judgment unit is used for: Based on the fusion features and the preset attention algorithm, the splicing prediction probability and the non-splicing prediction probability are obtained respectively. Based on the splicing prediction probability and the non-splicing prediction probability, determine whether to splice the first audio segment and the previous audio segment.

6. The system according to claim 1, characterized in that, The speech processing system also includes: a text adjustment module and a control module; The text adjustment module is used for: After splicing, the spliced ​​text is obtained; the splicing confidence score of the spliced ​​text is determined, the splicing confidence score includes acoustic coherence score and semantic logic score; When the splicing confidence score is less than the confidence threshold, the spliced ​​text is modified to obtain the modified spliced ​​text; The control module is used for: When the splicing confidence score is greater than or equal to the confidence threshold, a vehicle control command is generated based on the spliced ​​text; or, Based on the modified concatenated text, vehicle control commands are generated.

7. The system according to claim 1, characterized in that, The following two steps are executed simultaneously: The character recognition is performed based on the first feature information to obtain the first character recognition result; The first feature information is fused with the second feature information of the previous audio segment in the audio segment sequence to obtain the fused feature; Alternatively, the following two steps can be performed simultaneously: The character recognition is performed based on the first feature information to obtain the first character recognition result; Based on the fusion features, it is determined whether the first audio segment needs to be spliced ​​with the previous audio segment.

8. A speech processing method, characterized in that, The method includes: The first audio segment in the audio segment sequence is subjected to feature recognition to obtain the first feature information; Upon obtaining the first feature information, the text conversion mechanism and splicing judgment mechanism are triggered. According to the text conversion mechanism, character recognition is performed based on the first feature information to obtain the first character recognition result; According to the splicing judgment mechanism, the first feature information is fused with the second feature information of the previous audio segment of the first audio segment in the audio segment sequence to obtain a fused feature; based on the fused feature, it is determined whether the first audio segment and the previous audio segment need to be spliced. When the judgment result is yes, the first character recognition result and the second character recognition result corresponding to the previous audio segment are obtained and concatenated.

9. A vehicle, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the speech processing method of claim 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to perform the speech processing method of claim 8.

Images