Voice processing method and server

Abstract

The application relates to a voice processing method and a server. The method comprises the following steps: receiving a voice signal of an in-cabin sound area forwarded by a vehicle; combining voice algorithms with different functions to extract features of the voice signal, obtaining a feature extraction result and caching the feature extraction result; calling the feature extraction result in the cache and calculating through an acoustic model according to preset configuration parameters of each voice algorithm, to obtain corresponding acoustic calculation results; inputting each acoustic calculation result into a decoding module of the corresponding voice algorithm for decoding, and collecting each decoding result to obtain a voice processing result and send the voice processing result to the vehicle. The scheme provided by the application can make each voice algorithm extract features from the combined voice signal, avoid repeated operations, and reduce the occupation of CPU resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent voice, in particular to a voice processing method and a server. BACKGROUND

[0002] Voice interaction is a new generation of interaction mode based on voice input. With the continuous development of the automobile industry and human-computer interaction technology, intelligent vehicles also provide voice interaction functions for users.

[0003] In related technologies, intelligent vehicles can carry voice assistants to provide natural human-computer interaction. Users in the vehicle can not only control vehicle software such as navigation and music through voice, but also control vehicle hardware such as vehicle windows and air conditioners.

[0004] However, since voice applications have high real-time requirements, various voice algorithms such as voice recognition algorithms, voiceprint recognition algorithms, and voice wake-up algorithms involve large amounts of data processing. When multiple algorithms are processed simultaneously, they occupy a high CPU resource, which can easily cause other applications to lag and affect the data processing efficiency of other applications. SUMMARY

[0005] To solve or partially solve the problems in related technologies, the present application provides a voice processing method and a server, which can enable each voice algorithm to extract features from merged voice signals, avoid repeated operations, and reduce CPU resource occupation.

[0006] The first aspect of the present application provides a voice processing method, which includes: receiving a voice signal of an in-cabin sound area forwarded by a vehicle; merging voice algorithms with different functions to extract features of the voice signal, obtaining feature extraction results and caching them; calling the feature extraction results in the cache and calculating through an acoustic model according to preset configuration parameters of each voice algorithm, to obtain corresponding acoustic calculation results; inputting each acoustic calculation result into a decoding module of the corresponding voice algorithm for decoding, and summarizing each decoding result to obtain a voice processing result and send it to the vehicle. By merging voice algorithms to extract features from voice signals, repeated operations are avoided, and CPU resource occupation is reduced.

[0007] In the voice processing method of the present application, the merging of voice algorithms with different functions to extract features of the voice signal includes: in the corresponding sound area, merging voice algorithms with different functions to extract features in the voice signal according to a preset format. For different sound areas, feature extraction can be performed in the server independently and the format of corresponding feature extraction can be set independently, without interference, to meet the demand for multi-directional feature extraction.

[0008] In the voice processing method of the present application, the feature extraction results in the cache are called according to the preset configuration parameters of each voice algorithm, including: according to the preset configuration parameters of each voice algorithm, the feature extraction results in the cache corresponding to the frame number are spliced to generate the corresponding feature matrix; wherein the preset configuration parameters include at least one of the splicing frame number, the filling frame number, the filling behavior, and the interval frame number. The feature matrix input requirements of different acoustic models can be met according to the preset configuration parameters, which can be realized by simply setting the parameters, and the matrix splicing scheme is simplified.

[0009] In the voice processing method of the present application, the feature extraction results in the cache are called according to the preset configuration parameters of each voice algorithm, including: according to the preset configuration parameters of each voice algorithm, the feature extraction results in the cache corresponding to the frame number are spliced to generate the corresponding feature matrix; wherein the preset configuration parameters include at least one of the splicing frame number, the filling frame number, the filling behavior, and the interval frame number. The feature matrix input requirements of different acoustic models can be met according to the preset configuration parameters, which can be realized by simply setting the parameters, and the matrix splicing scheme is simplified.

[0010] In the voice processing method of the present application, before the acoustic calculation results are respectively input into the decoding module of the corresponding voice algorithm for decoding, it further includes: respectively determining whether the corresponding voice algorithm obtains the feature matrix; if it is determined that the voice algorithm obtains the corresponding feature matrix, the acoustic calculation result output by the acoustic model is decoded. By judging whether the front-end feature matrix is valid, it is further determined whether decoding is needed, which avoids invalid mobilization of the decoding module and saves CPU resources.

[0011] In the voice processing method of the present application, the voice signal of the in-cabin sound area forwarded by the vehicle is received, including: respectively receiving the voice signal of each sound area in the in-cabin sound area forwarded by the vehicle; the features of the voice signal are extracted by combining different functions of the voice algorithm, including: according to different functions of the voice algorithm, the features of the voice signal are extracted by combining the voice signals from the same sound area. The server distinguishes the voice signal processing of different sound areas, and the voice signal corresponding to each sound area is only extracted once, which saves CPU resources, avoids interference, and ensures the accuracy of the final voice processing result.

[0012] In the voice processing method, the calculating of the acoustic model comprises: invoking corresponding acoustic sub-models according to a calculation interface of a preset neural network processing engine to obtain acoustic calculation results of corresponding voice algorithms. By transplanting the acoustic model to the professional neural network processing engine for calculation, the occupation of CPU resources is reduced, and different voice algorithms have their own acoustic sub-models, which are calculated independently and in parallel, thereby improving the calculation efficiency.

[0013] In the voice processing method, the method further comprises: when the number of sound zones in the cabin is multiple, pre-setting acoustic models corresponding to each of the sound zones or acoustic models corresponding to the sound zones collectively. By independently setting or collectively setting acoustic models for each sound zone in the server, a flexible model architecture is achieved.

[0014] The second aspect of the application provides a server, comprising: an information transceiving module configured to receive a voice signal of a sound zone in a cabin forwarded by a vehicle; a feature extraction module configured to merge and extract features of the voice signal according to different voice algorithms to obtain a feature extraction result and cache the feature extraction result; an acoustic model calculation module configured to invoke the cached feature extraction result according to preset configuration parameters of each voice algorithm and calculate the corresponding acoustic calculation result by using an acoustic model; and a decoding module configured to decode each acoustic calculation result and aggregate each decoding result to obtain a voice processing result, and send the voice processing result to the vehicle through the information transceiving module. The server of the application can reduce the occupation of CPU resources while improving the voice processing efficiency.

[0015] The third aspect of the application provides a server, comprising:

[0016] a processor; and

[0017] a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method described above.

[0018] The fourth aspect of the application provides a computer-readable storage medium having executable code stored thereon, which, when executed by a processor of a server, causes the processor to perform the method described above.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other objects, features and advantages of this application will become more apparent from the following description of exemplary embodiments of this application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of this application.

[0021] Figure 1 This is a schematic flowchart of the speech processing method shown in this application;

[0022] Figure 2 This is a schematic flowchart of the speech processing method shown in this application;

[0023] Figure 3 This is a schematic diagram of the server structure shown in this application;

[0024] Figure 4 This is a schematic diagram of the server structure shown in this application;

[0025] Figure 5 This application illustrates the structural diagram of a server in a single audio region;

[0026] Figure 6 This is a schematic diagram of the server structure shown in this application. Detailed Implementation

[0027] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While embodiments of this application are shown in the drawings, it should be understood that this application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make this application more thorough and complete, and to fully convey the scope of this application to those skilled in the art.

[0028] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0029] It should be understood that although the terms "first," "third," "third," etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as third information, and similarly, third information may also be referred to as first information. Thus, a feature defined as "first" or "third" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0030] In related technologies, when multiple functional algorithms such as speech recognition, voiceprint recognition, and voice wake-up process the same voice data simultaneously, they will occupy high CPU resources and easily cause lag in other applications running at the same time.

[0031] To address the aforementioned issues, this application provides a speech processing method that enables various speech algorithms to extract common features from speech signals, avoiding repetitive operations and reducing CPU resource consumption.

[0032] The technical solution of this application is described in detail below with reference to the accompanying drawings.

[0033] Figure 1 This is a schematic flowchart of the speech processing method shown in this application.

[0034] See Figure 1 This application discloses a speech processing method, which includes:

[0035] S110 receives the voice signal from the cabin audio zone relayed by the vehicle.

[0036] The number of audio zones within a vehicle's cabin may be one or more. In this step, when the server is the executing entity, it receives voice signals from different audio zones. For example, it receives voice signals from different audio zones within the vehicle's cabin, such as the driver's seat zone, passenger's seat zone, rear left-side rear right-side rear-side rear-side zone, front-seat zone, and rear-seat zone. The audio zone settings within the vehicle's cabin are only illustrated here and are not limited. It can be understood that each audio zone is set independently, and the server processes the received voice signals from different zones independently.

[0037] S120 combines different speech algorithms to extract features from the speech signal, obtains the feature extraction results, and caches them.

[0038] To perform a more comprehensive analysis of the speech signal, different speech algorithms with different functions need to be used for processing. These algorithms can be, for example, speech recognition algorithms, voice wake-up algorithms, and voiceprint recognition algorithms, and are not limited to these. Each speech algorithm needs to extract features from the speech signal to achieve its corresponding function. To reduce CPU resource consumption, in this step, the speech algorithms for each function do not need to extract features from the speech signal separately; that is, the feature extraction process can be combined. Only one feature extraction is needed, eliminating the need for repeated feature extraction. The obtained feature extraction results can be applied to the speech algorithms for each function, thereby reducing CPU computing resources.

[0039] Furthermore, in this step, for example, related techniques can be used to extract speech feature vectors such as MFCC (Mel Frequency Cepstral Coefficents, cepstral parameters extracted in the Mel-scale frequency domain) or FBank (Filterbanks, acoustic features obtained by applying Mel filter banks to the energy spectrum), thus obtaining the feature extraction result.

[0040] Upon receiving the voice signal relayed from the vehicle, feature extraction can be performed in real time to ensure timeliness. The feature extraction results are then cached so that subsequent steps can utilize feature extraction results from different frames based on the computational needs of different voice algorithms. Essentially, the cache stores feature extraction results corresponding to more than one frame of voice signal.

[0041] S130, according to the preset configuration parameters of each speech algorithm, calls the feature extraction results in the cache and calculates them through the acoustic model to obtain the corresponding acoustic calculation results.

[0042] In this step, each speech algorithm performs different functions and requires different acoustic models for calculation. Therefore, the number of feature extraction result frames required by each acoustic model may differ. Based on this, each speech algorithm has corresponding preset configuration parameters. Using the feature extraction results stored in the cache in time sequence, the feature extraction results for the corresponding number of frames can be processed into the input data required by the acoustic model of each speech algorithm, according to the preset configuration parameters of each speech algorithm.

[0043] Specifically, preset configuration parameters can include the number of frames to be stitched, the number of frames to be filled, the filling behavior, and the number of frames to be spaced. Based on these configuration parameters, the feature extraction results of the corresponding number of frames can be retrieved from the cache for processing to form a feature matrix. This allows the corresponding acoustic model to obtain the input data required by the acoustic model, enabling each acoustic model to calculate the acoustic calculation results based on the feature matrix.

[0044] Different speech algorithms produce different acoustic calculation results from their corresponding acoustic models. For example, a speech algorithm for speech recognition can obtain a sequence of phoneme features and their corresponding probabilities through the corresponding acoustic model; a speech algorithm for voice wake-up can obtain a sequence of phoneme features of the wake word and their corresponding probabilities through the corresponding acoustic model; and a speech algorithm for voiceprint recognition can obtain a sequence of speaker features and their corresponding probabilities through the corresponding acoustic model. These are just examples and do not limit the types of acoustic calculation results that each acoustic model will produce in practical applications.

[0045] S140 inputs each acoustic calculation result into the corresponding speech algorithm decoding module for decoding, summarizes each decoding result, obtains the speech processing result, and sends it to the vehicle.

[0046] In this step, the speech algorithms for each function are decoded using different decoding modules. For example, in related technologies, the speech algorithm for speech recognition mainly performs Viterbi decoding on the posterior probability calculated by the acoustic model on the corresponding decoding graph; the speech algorithm for voice wake-up mainly performs Viterbi decoding on the posterior probability calculated by the acoustic model on the corresponding decoding graph; and the speech algorithm for voiceprint recognition mainly decodes by calculating cosine similarity. This is only an example, and the actual decoding scheme of each decoding module is not limited here.

[0047] By inputting the acoustic calculation results into the corresponding decoding modules for decoding, the decoding results output by each decoding module are obtained. It is understandable that, due to the different running times and durations of each decoding module, some modules may fail to start decoding due to a lack of valid input data. To ensure timely user experience, the decoding results obtained at the current time can be summarized to generate corresponding voice processing results and sent to the vehicle promptly.

[0048] It can be understood that, based on the speech signals from different vocal regions, the above steps S120 to S140 are processed separately to obtain the corresponding speech processing results. That is, the server processes the speech signals from different vocal regions independently.

[0049] As can be seen from this example, the speech processing method of this application receives speech signals from each voice region forwarded by the vehicle, merges different speech algorithms to extract the speech signals from each voice region, eliminating the need for each speech algorithm to extract features from the speech signal separately, reducing repetitive operations and thus reducing CPU resource consumption. In addition, by simply adjusting the preset configuration parameters, different feature extraction results in the cache can be used as input, and acoustic calculation results can be obtained according to the corresponding acoustic model, making it easy to extend the speech algorithm with new functions. At the same time, the decoding modules of each speech algorithm perform decoding independently without waiting, ensuring the real-time output of decoding results, improving CPU resource utilization, and finally summarizing the real-time generated decoding results and sending them to the vehicle in a timely manner, improving the user experience.

[0050] Figure 2 This is a schematic flowchart of the speech processing method shown in this application.

[0051] See Figure 2 This application discloses a speech processing method, which includes:

[0052] S210 receives voice signals from each audio zone in the cockpit relayed by the vehicle.

[0053] In this speech processing method, the executing entity can be a server, which can receive speech signals relayed by the vehicle in real time. Based on the independence of each voice region, each voice region acquires speech signals through its corresponding microphone and relays them from the vehicle to the server in real time. The server can receive speech signals from different voice regions simultaneously.

[0054] S220, based on different speech algorithms, merges speech signals originating from the same vocal range, extracts the features of the speech signals, obtains the feature extraction results, and caches them.

[0055] The server can pre-load different speech algorithm engines, such as speech recognition, voiceprint recognition, and voice wake-up engines. Each speech algorithm engine shares the same feature extraction module, which is mapped to a specific vocal region. Features are extracted from speech signals within the same vocal region using this same module; the same speech signal only needs to undergo feature extraction once, eliminating the need for repetition. Feature extraction results from different vocal regions can be cached separately to avoid confusion.

[0056] Furthermore, within the corresponding audio regions, different speech algorithms are merged according to a preset format to extract features from the speech signal. That is, for a speech signal from a single audio region, when various speech algorithms are merged using the same feature extraction module for feature extraction, a preset format of feature extraction results can be obtained. For speech signals from different audio regions, preset formats for the extracted feature results can be set separately. For example, the preset format of the feature extraction results could be an MFCC (Mel Frequency Cepstral Coefficents, cepstral parameters extracted in the Mel-scale frequency domain) feature vector or an FBank (Filterbanks, acoustic features obtained by applying Mel filter banks to the energy spectrum) feature vector. For example, the feature extraction results corresponding to each audio region can uniformly use 80-dimensional first-order FBank features; this is merely an example, and the feature extraction results corresponding to different audio regions can use the same or different preset formats.

[0057] S230, based on the preset configuration parameters of each speech algorithm, calls the feature extraction results of the corresponding frame number in the cache for concatenation to generate the corresponding feature matrix.

[0058] The preset configuration parameters for each speech algorithm include at least one of the following: number of splicing frames, number of padding frames, padding behavior, and interval frames. The number of splicing frames refers to the number of spliced ​​speech frames after segmentation; the number of padding frames refers to the number of padding frames to the left or right of the speech frames; the padding behavior includes zero padding or duplication; and the interval frames refer to the number of interval frames between two feature frames. Other configuration parameters may also be included, without limitation. It can be understood that by setting specific values ​​for the preset configuration parameters corresponding to different speech algorithms, specific splicing behaviors can be defined, thereby obtaining different feature matrices to meet the feature input format requirements of the acoustic models of different speech algorithms. Optionally, by adjusting the configuration parameters, new speech algorithms can be easily extended, enriching the functional modules of human-computer interaction.

[0059] Optionally, before concatenating the feature extraction results from the same vocal region, CMVN (Cepstral Mean and Variance Normalization) can be used to normalize each feature extraction result (e.g., speech feature vector). Specifically, after obtaining the speech feature vector, each speech feature vector is transformed from one space to another, making the feature parameters in this space more consistent with a certain probability distribution, compressing the dynamic range of the feature parameter value domain, and obtaining more standard speech feature vectors. This helps to improve the robustness of the prediction results of the acoustic model in subsequent steps.

[0060] Furthermore, upon receiving a new speech signal, the feature extraction results in the cache are called according to the feature concatenation interface. When the feature extraction results meet the preset configuration parameters of at least one speech algorithm, the corresponding feature matrix is ​​generated according to the concatenation function in the feature concatenation interface. Specifically, a feature concatenation interface can be set in the feature extraction module corresponding to each voice region, and a concatenation function can be predefined in each feature concatenation interface. The concatenation function is used to call the feature extraction results of the corresponding frame number in the cache according to the preset configuration parameters of each speech algorithm, and concatenate the feature extraction results into the feature matrix required by the corresponding acoustic model.

[0061] In other words, based on streaming computing, as the server receives new speech signals in real time, the feature extraction module correspondingly extracts features from the speech signals and generates feature extraction results in real time. Since the number of frames of feature extraction results generated in the cache at any given moment is uncertain, when the number of frames of feature extraction results in the cache meets the preset configuration parameters of any speech algorithm, the feature extraction results of an appropriate number of frames can be concatenated into the corresponding feature matrix through the concatenation function in the feature concatenation interface, thus obtaining a valid feature matrix. This valid feature matrix can then be input into the corresponding acoustic model for calculation in subsequent steps.

[0062] S240: According to the calculation interface of the preset neural network processing engine, each speech algorithm calls the corresponding acoustic sub-model and performs calculations based on the input feature matrix to obtain the acoustic calculation results of the corresponding speech algorithm.

[0063] It's important to understand that to meet the real-time requirements of speech technology, the acoustic models of multiple speech algorithms involve massive computational demands, thus consuming significant CPU resources. To reduce CPU resource consumption, the acoustic models corresponding to each speech algorithm can be uniformly ported to a neural network processing engine, such as SNPE (Snapdragon Neural Processing Engine, a neural network processing engine provided by Qualcomm chips). This is merely an example. In other words, the acoustic models of each speech algorithm are integrated into a unified acoustic model within the neural network processing engine, and the acoustic models of each speech algorithm are equivalent to acoustic sub-models within this unified acoustic model. Therefore, the acoustic models of each speech algorithm mentioned in the previous steps are considered acoustic sub-models in this step. The neural network processing engine has a computational interface; the algorithm engines of each speech algorithm select the corresponding acoustic sub-model for computation by calling the same computational interface, which helps reduce CPU resource consumption.

[0064] It's important to understand that, due to the different input feature matrices, the acoustic sub-models of different speech algorithms may have partially identical or completely different computation frequencies. For example, the acoustic sub-model of a speech recognition algorithm might compute every n milliseconds, while the acoustic sub-model of a speaker recognition algorithm might compute every m milliseconds. Despite the different computation frequencies, each speech algorithm can attempt to call the acoustic sub-model for computation whenever the server receives a new speech signal, thereby improving timeliness through streaming computation. Specifically, the cache contains feature extraction results generated in a time-series manner. Before calling the acoustic sub-model for computation, each speech algorithm can first call the feature concatenation interface to attempt to concatenate the feature extraction results using the concatenation function. If the feature extraction results meet the corresponding preset configuration parameters, the concatenation function can concatenate them to generate the corresponding feature matrix, allowing the acoustic sub-model to perform computation based on the corresponding feature matrix and obtain the acoustic computation result. In other words, as long as the feature concatenation interface obtains a valid feature matrix based on any preset configuration parameters, it can input that feature matrix into the corresponding acoustic sub-model for computation without waiting for the computation results of other acoustic sub-models.

[0065] Preferably, when there are multiple sound zones in the cockpit, an acoustic model corresponding to each sound zone or a common acoustic model for multiple sound zones is pre-set in the server. For example, each sound zone is mapped to a single acoustic model in the server. After the server receives the speech signals from each sound zone, it performs the corresponding acoustic calculations using the corresponding acoustic model; wherein, the acoustic model includes acoustic sub-models corresponding to different speech algorithms. Alternatively, all sound zones are uniformly mapped to a single acoustic model in the server, which also includes acoustic sub-models corresponding to different speech algorithms, with each acoustic sub-model responsible for the acoustic calculations corresponding to its respective sound zone. By flexibly setting the number of acoustic models, the needs of different server architectures can be met.

[0066] S250 inputs the acoustic calculation results into the corresponding speech algorithm's decoding module for decoding to obtain the corresponding decoding results.

[0067] It is understandable that the frequencies of acoustic calculation results obtained by different speech algorithms may differ. Based on this, it is determined whether the corresponding speech algorithm has obtained a feature matrix. If it is determined that the speech algorithm has obtained the corresponding feature matrix, then decoding is performed based on the acoustic calculation results output by the acoustic model. That is, based on the feature concatenation interface in step S230, it can be determined whether a feature matrix corresponding to any speech algorithm has been generated. Subsequently, it can be determined whether an acoustic sub-model has performed calculations in step 240, thereby determining whether the acoustic sub-model has output acoustic calculation results. Then, the corresponding speech algorithm engine calls the corresponding decoding module for decoding.

[0068] In other words, if the feature concatenation interface in step S230 does not generate any feature matrix, the acoustic sub-model in step S240 cannot and does not need to perform calculations due to the lack of input data, and obviously cannot output acoustic calculation results. At this time, the algorithm engines of each speech algorithm do not need to call any decoding module for decoding. Determining whether to call the decoding module in this way helps reduce CPU resource consumption and avoids invalid calls.

[0069] S260 summarizes the decoding results, obtains the voice processing results, and sends them to the vehicle.

[0070] Based on streaming computing, the decoding results corresponding to each speech algorithm are aggregated to obtain the speech processing result. In other words, the content of the speech processing result obtained at different times differs. To ensure timeliness, the speech processing result obtained at the current time can be sent to the vehicle in real time, allowing the vehicle to provide feedback to the user based on the speech processing results.

[0071] As can be seen from this example, the speech processing method of this application allows the server to receive and process speech signals from different vocal ranges independently. By employing corresponding feature extraction modules, the speech algorithms can merge and extract features without repeated feature extraction, reducing CPU resource consumption. Furthermore, by using different preset configuration parameters to meet the splicing requirements of each speech algorithm, the corresponding feature matrix is ​​obtained from the buffer through the feature splicing interface. Simultaneously, by obtaining the feature matrix corresponding to any speech algorithm, effective calculations can be performed through the corresponding acoustic sub-model to obtain the corresponding acoustic calculation result, and the corresponding decoding module can be called for decoding, thus avoiding unnecessary CPU resource consumption and reducing lag caused by CPU resource consumption on other applications. In addition, integrating the acoustic models of each speech algorithm as acoustic sub-models into the same neural network processing engine avoids scattered settings and also reduces CPU resource consumption.

[0072] Corresponding to the aforementioned application function implementation method embodiments, this application also provides a server and corresponding embodiments.

[0073] Figure 3 This is a schematic diagram of the server structure shown in this application.

[0074] See Figure 3 The server shown in this application includes an information transceiver module 310, a feature extraction module 320, an acoustic model calculation module 330, and a decoding module 340. Wherein:

[0075] The information transceiver module 310 is used to receive voice signals from the cabin audio zone forwarded by the vehicle.

[0076] The feature extraction module 320 is used to merge different speech algorithms to extract features of speech signals, obtain feature extraction results, and cache them.

[0077] The acoustic model calculation module 330 is used to call the feature extraction results in the cache and calculate them through the acoustic model according to the preset configuration parameters of each speech algorithm to obtain the corresponding acoustic calculation results.

[0078] The decoding module 340 is used to decode each acoustic calculation result separately, summarize each decoding result to obtain the speech processing result, and send the speech processing result to the vehicle through the information transceiver module.

[0079] Figure 4 This is another schematic diagram of the server structure shown in this application. Figure 5 This is a schematic diagram of the server's structure in a single audio region, as shown in this application.

[0080] See Figure 4 and Figure 5 Specifically, the information transceiver module 310 is used to receive voice signals from various audio zones within the cabin, relayed by the vehicle. The feature extraction module 320 is used to merge voice signals originating from the same audio zone and extract features from the voice signals according to different functional voice algorithms. There may be one or more feature extraction modules 320, each processing a voice signal from one audio zone. Voice algorithms of different functions share a single feature extraction module to extract features from the voice signals of the same audio zone. Optionally, within a corresponding audio zone, the feature extraction module 320 is used to merge voice algorithms of different functions according to a preset format to extract features from the voice signals.

[0081] The acoustic model calculation module 330 includes a feature concatenation module 331 and a calculation module 332. The feature concatenation module 331 is used to concatenate the feature extraction results of corresponding frames in the cache according to the preset configuration parameters of each speech algorithm, generating a corresponding feature matrix. The preset configuration parameters include at least one of the following: concatenation frame number, padding frame number, padding behavior, and interval frame number. Optionally, before performing feature concatenation, the feature concatenation module 331 performs CMVN normalization processing on the feature extraction results.

[0082] Furthermore, the feature splicing module 331 is used to call the feature extraction results in the cache according to the feature splicing interface when a new speech signal is received; when the feature extraction results meet the preset configuration parameters of at least one speech algorithm, the corresponding feature matrix is ​​generated according to the splicing function in the feature splicing interface. Optionally, the number of feature splicing modules 331 corresponds to the number of each functional speech algorithm, that is, each feature splicing module 331 has its own splicing function, which is used to splice the feature matrix required by the acoustic model of the corresponding speech algorithm. Alternatively, each speech algorithm can share a feature splicing module 331, so that the same feature splicing module 331 splices the feature matrix according to different splicing functions.

[0083] Furthermore, when there are multiple sound zones within the cockpit, an acoustic model corresponding to each individual sound zone or an acoustic model shared by multiple sound zones is pre-set. Each acoustic model includes an acoustic sub-model corresponding to a specific algorithm. The calculation module 332 is used to call the corresponding acoustic sub-models for calculation according to the calculation interface of the preset neural network processing engine, thereby obtaining the acoustic calculation results of the corresponding speech algorithm.

[0084] The number of decoding modules 340 corresponds to the number of different speech algorithms. Before decoding, each decoding module 340 calls the feature concatenation interface of the feature concatenation module to determine whether the corresponding speech algorithm has obtained a feature matrix. If the speech algorithm has obtained the corresponding feature matrix, decoding is performed based on the acoustic calculation results output by the acoustic model. This design avoids unnecessary calls to the decoding modules 340 and saves CPU resources.

[0085] In summary, the server of this application can extract features from speech signals of the same audio range forwarded by vehicles in a common feature extraction module, eliminating the need for repeated feature extraction and saving CPU resources. The feature concatenation module obtains the feature matrices required by the corresponding speech algorithms according to different preset configuration parameters. The corresponding acoustic sub-models are called through the unified calculation interface of the calculation module to obtain the corresponding acoustic calculation results. In addition, the decoding module performs acoustic model calculation and decoding in real time according to the feature matrix in the feature concatenation module, meeting the timeliness requirements of speech technology, while avoiding invalid calls to the decoding module and reducing the occupation of CPU resources. Each decoding module runs its corresponding decoding independently, improving processing efficiency and shortening the CPU occupation time through parallel decoding.

[0086] Regarding the server in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated further here.

[0087] Figure 6 This is a schematic diagram of the server structure shown in this application.

[0088] See Figure 6 The server 1000 includes a memory 1010 and a processor 1020.

[0089] The processor 1020 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0090] Memory 1010 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by processor 1020 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 1010 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1010 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, a high-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0091] The memory 1010 stores executable code, which, when processed by the processor 1020, can cause the processor 1020 to execute part or all of the methods described above.

[0092] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0093] Alternatively, this application may be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium) storing executable code (or computer program or computer instruction code) thereon, which, when executed by a processor of a server (or server, etc.), causes the processor to perform part or all of the steps of the above-described method according to this application.

[0094] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A speech processing method, characterized in that, include: Receives voice signals from the cabin audio zone relayed by the vehicle; Different speech algorithms with different functions are combined to extract features from speech signals, and the feature extraction results are obtained and cached. When the number of frames of the feature extraction results in the cache meets the preset configuration parameters of any speech algorithm, the feature extraction results of the corresponding number of frames in the cache are called and spliced ​​through the splicing function in the feature splicing interface to generate the corresponding feature matrix, and the corresponding acoustic calculation result is obtained through the acoustic model calculation; wherein, the preset configuration parameters include at least one of splicing frame number, filling frame number, filling behavior, and interval frame number; Each speech algorithm is determined to have obtained a feature matrix. If the speech algorithm has obtained a feature matrix, the corresponding acoustic calculation result is input into the decoding module of the corresponding speech algorithm for decoding. The decoding results of each speech algorithm are then summarized to obtain the speech processing result and sent to the vehicle.

2. The method according to claim 1, characterized in that, The method of merging different speech algorithms to extract features from speech signals includes: In the corresponding audio region, different voice algorithms are merged according to a preset format to extract features from the voice signal; The acoustic model includes multiple acoustic sub-models, and each speech algorithm has its own corresponding acoustic sub-model; wherein, the acoustic sub-model is used to calculate the corresponding acoustic calculation result based on the feature matrix corresponding to the speech algorithm.

3. The method according to claim 1, characterized in that, The receiving vehicle relays the voice signal from the cabin audio zone, including: It receives voice signals from each audio zone in the cabin, relayed by the vehicle. The method of merging different speech algorithms to extract features from speech signals includes: Based on different speech algorithms, speech signals originating from the same vocal range are merged to extract the features of the speech signals.

4. The method according to claim 1, characterized in that, The process of obtaining corresponding acoustic calculation results through acoustic model calculations includes: Based on the calculation interface of the preset neural network processing engine, the corresponding acoustic sub-models are called to perform calculations and obtain the acoustic calculation results of the corresponding speech algorithm.

5. The method according to claim 4, characterized in that, The method further includes: When there are multiple sound zones in the cockpit, an acoustic model corresponding to each individual sound zone or an acoustic model corresponding to multiple sound zones is preset.

6. A server, characterized in that, include: The information transceiver module is used to receive voice signals from the cabin audio zone forwarded by the vehicle. The feature extraction module is used to merge different speech algorithms to extract features from the speech signal, obtain the feature extraction results, and cache them. The acoustic model calculation module is used to, when the number of frames of the feature extraction results in the cache meets the preset configuration parameters of any speech algorithm, call the feature extraction results of the corresponding number of frames in the cache through the splicing function in the feature splicing interface to splice them, generate the corresponding feature matrix, and obtain the corresponding acoustic calculation result through acoustic model calculation; wherein, the preset configuration parameters include at least one of splicing frame number, filling frame number, filling behavior, and interval frame number; The decoding module is used to determine whether the corresponding speech algorithm has obtained the feature matrix. If it is determined that the speech algorithm has obtained the corresponding feature matrix, the corresponding acoustic calculation result is input into the decoding module of the corresponding speech algorithm for decoding, and the decoding results of each speech algorithm obtained at the moment are summarized to obtain the speech processing result. The speech processing result is then sent to the vehicle through the information transceiver module.

7. A server, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium having executable code stored thereon, which, when executed by a processor of a server, causes the processor to perform the method as described in any one of claims 1-5.

Images