Audio processing method and system based on vehicle, vehicle, storage medium and product

Abstract

The invention provides an audio processing method and system based on a vehicle, the vehicle, a storage medium and a product, applied to the vehicle, the vehicle comprises an audio processing device configured with a plurality of pulse code modulation (PCM) nodes and a plurality of audio processing processes; the method comprises the steps that original audio data for a target vehicle is acquired, the original audio data comprises multiple audio contents, and each audio content corresponds to different audio processing purposes in the target vehicle; multiple paths of audio data streams are generated according to the original audio data, each path of audio data stream is written into different PCM nodes, and the audio content of each path of audio data stream is the same as that of the original audio data; the audio data stream is read from the target PCM node through a target audio processing process, audio processing operation corresponding to the binding purpose is executed, and the target audio processing process is matched with the audio processing purpose of the target PCM node.

Description

Technical Field

[0001] This invention relates to the automotive field, and more particularly to a vehicle-based audio processing method, system, vehicle, storage medium, and product. Background Technology

[0002] With the development of smart cockpit technology, user demand for audio functions such as microphone-free karaoke and in-car voice assistant interaction is increasing. However, these functions require exclusive access to microphone resources. This strong dependence on a single physical resource leads to inherent mutual exclusion conflicts between different functions. When users expect to use multiple functions simultaneously, the vehicle's infotainment system cannot meet such collaborative operation requirements, severely restricting the integrity of smart cockpit services and the consistency of the user experience.

[0003] In related technologies, in-vehicle infotainment systems typically employ a function mutual exclusion management mechanism as a solution. Specifically, when a specific function that requires exclusive microphone resources is activated, this mechanism will forcibly interrupt or disable other functional modules that also rely on microphone input, thereby ensuring that the currently running function receives interference-free audio input. However, users will be forced to make single selections among key capabilities such as entertainment, security monitoring, and convenient interaction, which will limit the overall intelligent service capabilities of the cockpit. Summary of the Invention

[0004] In view of this, the present invention provides a vehicle-based audio processing method, system, vehicle, readable storage medium, and program product to address the shortcomings of related technologies.

[0005] Specifically, this specification is implemented through the following technical solution:

[0006] According to a first aspect of this specification, an audio processing method is provided, the method comprising:

[0007] Acquire raw audio data for the target vehicle, the raw audio data including multiple audio contents, each audio content corresponding to a different audio processing purpose in the target vehicle;

[0008] Multiple audio data streams are generated based on the original audio data, and each audio data stream is written to a different PCM node. The audio content of each audio data stream is the same as the original audio data.

[0009] The target audio processing process reads the audio data stream from the target PCM node and performs the audio processing operation corresponding to the binding purpose, wherein the target audio processing process is matched with the audio processing purpose of the target PCM node.

[0010] According to a second aspect of this specification, an audio processing system is provided for use in a vehicle, the vehicle including an audio processing device configured with multiple pulse code modulation (PCM) nodes and multiple audio processing processes, each PCM node having a corresponding different audio processing purpose, and each audio processing process being bound to a different audio processing purpose; wherein:

[0011] The audio processing device is used to acquire raw audio data for a target vehicle, the raw audio data including multiple audio contents, each audio content corresponding to a different audio processing purpose in the target vehicle; and to generate multiple audio data streams based on the raw audio data, and write each audio data stream to a different PCM node, the audio content of each audio data stream being the same as the raw audio data.

[0012] The audio processing process is used to read audio data streams from target PCM nodes that match the audio processing purpose, and to perform audio processing operations corresponding to the binding purpose.

[0013] According to a third aspect of this specification, a vehicle is provided, comprising: a processor; a memory for storing processor-executable instructions; and a voice input device; wherein the processor performs the steps of the method as described in the first aspect by executing the executable instructions.

[0014] According to a fourth aspect of this specification, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0015] According to a fifth aspect of this specification, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] This manual describes how, after acquiring the raw audio data for the target vehicle, multiple audio data streams with identical audio content can be generated based on the raw audio data. These streams are then written into Pulse Code Modulation (PCM) nodes corresponding to different audio processing purposes. This allows each audio processing process to read the audio data stream in parallel from its dedicated PCM node, which matches its specific audio processing purpose, and perform the corresponding audio processing operations. As a result, users can use multiple audio functions simultaneously without changing the actual microphone resources of the target vehicle, ensuring the overall intelligent service capabilities of the cockpit. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the present invention, the accompanying drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0018] Figure 1 This is a schematic diagram of the architecture of a vehicle-based audio processing system as shown in an embodiment of the present invention;

[0019] Figure 2 This is a flowchart illustrating a vehicle-based audio processing method according to an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of the transmission path of raw audio data shown in an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of another vehicle-based audio processing system architecture shown in the disclosed embodiments of the present invention;

[0022] Figure 5 This is a schematic diagram illustrating an audio management configuration scheme according to an embodiment of the present invention;

[0023] Figure 6 This is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

[0024] Figure 7 This is a block diagram illustrating a vehicle-based audio processing device according to an embodiment of the present invention. Detailed Implementation

[0025] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present invention.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0027] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0028] The embodiments of the vehicle-based audio processing method of the present invention will now be described in detail with reference to the accompanying drawings.

[0029] Figure 1 This is a schematic diagram illustrating the architecture of a vehicle-based audio processing system according to embodiments disclosed in this specification. Figure 1 As shown, the system may include a voice input device 12 and an audio processing device 14 in the target vehicle 10.

[0030] The voice input device 12 can be used as an electronic device for acquiring raw audio data from inside the vehicle or related areas, that is, it can be Figure 1 The independent microphone or microphone array mounted on the target vehicle 10 shown can also be a user terminal device or other external device that supports voice acquisition and is connected to the target vehicle 10. Figure 1 This can be manifested as the voice input device 12 being an independent device outside the target vehicle 10. The aforementioned user terminal can be a mobile phone, tablet, laptop, PDA (Personal Digital Assistants), wearable device (such as smart glasses, smartwatches, etc.), and one or more embodiments of this specification do not limit this. Specifically, the raw audio data collected by the voice input device 12 is not for a single purpose, but can contain multiple different types of audio content. These different types of audio content correspond to different audio application scenarios or processing purposes within the vehicle that require independent processing, such as driver voice command recognition, passenger communication, in-vehicle entertainment system control, and active noise cancellation reference signal input. In short, each type of audio content corresponds to a different audio processing purpose within the target vehicle 10, providing basic data input for subsequent audio processing flows.

[0031] The audio processing device 14 can receive and process raw audio data from the voice input device 12. It supports generating multiple audio data streams with identical audio content based on any acquired raw audio data. Simultaneously, these audio data streams with identical content but independent paths are written into n different pulse code modulation (PCM) nodes according to their respective preset audio processing purposes. Each PCM node can specifically serve a particular audio processing purpose. For example, assuming a vehicle is currently acquiring audio data through the same onboard microphone array, and this audio data contains a driver's voice-based voice recognition command and KTV singing, two audio data streams with identical content can be generated, one for voice recognition and one for KTV singing. The audio data stream used for voice recognition is written to the PCM node also used for voice recognition, and the audio data stream used for KTV singing is written to the same PCM node. Furthermore, the audio processing device 14 can run m independent audio processing processes, each of which can be pre-bound to one or more specific audio processing purposes, where n and m are both positive integers. These processes proactively read the corresponding audio data streams from specific PCM nodes that are bound to them and strictly match their processing purpose. Once the data is read, the process immediately executes the audio processing operations required for its specific purpose, ultimately achieving parallel and targeted processing for different audio processing purposes.

[0032] Figure 2 This is a flowchart illustrating an exemplary embodiment of the present invention for a vehicle-based audio processing method. The method is applied to a vehicle, which includes an audio processing device configured with multiple pulse code modulation (PCM) nodes and multiple audio processing processes. Each PCM node has a corresponding different audio processing purpose, and each audio processing process is bound to a different audio processing purpose. Specifically, it may include the following steps:

[0033] Step S202: Obtain raw audio data for the target vehicle. The raw audio data includes multiple audio contents, each of which corresponds to a different audio processing purpose in the target vehicle.

[0034] The audio processing device first acquires raw audio signals through a voice input device associated with the target vehicle. This raw audio data can contain audio content for various purposes, such as commands for voice recognition, human voices for calls, and ambient sound monitoring sounds present in the vehicle's cabin. Each type of content corresponds to different subsequent audio processing applications within the vehicle. These audio processing applications can include at least two of the following: voice assistant interaction processing, in-vehicle entertainment system audio processing, in-vehicle call processing, voiceprint-based biometric recognition, driver state analysis based on audio features, recording function processing, occupant emotion recognition, in-vehicle active noise cancellation processing, and cabin sound field optimization processing. In short, the core objective of all these audio processing applications is to perform specific functional processing or information extraction on the raw audio data acquired within the vehicle. This specification does not limit the specific types of functions.

[0035] Step S204: Generate multiple audio data streams based on the original audio data, and write each audio data stream to a different PCM node. The audio content of each audio data stream is the same as the original audio data.

[0036] After acquiring the raw audio data, the data can be split and assigned to PCM nodes. Specifically, the system can generate multiple audio data streams with identical content based on the same raw audio data. Assuming the raw audio data contains two types of audio content, each of the multiple audio data streams will also contain both types of audio content. These parallel data streams can then be guided and written to different PCM nodes in the Hardware Abstraction Layer (HAL). Each PCM node is pre-assigned to a specific audio processing purpose, such as a KTV recording PCM node, a speech recognition PCM node, a call PCM node, or a noise reduction PCM node, thus achieving preliminary physical or logical separation and routing of the audio data streams according to their final processing purpose.

[0037] Regarding the above-mentioned audio data stream generation process, the solution in this specification can introduce, for example... Figure 1 It is implemented using a Digital Signal Processor (DSP).

[0038] In one embodiment, such as Figure 3As shown, the aforementioned audio processing device can input the raw audio data collected by the vehicle-mounted microphone, which serves as a voice input device, into a DSP chip at the same hardware layer, based on a preset target format and using a Time Division Multiplexing (TDM) communication architecture. The DSP chip can be configured with a dedicated demultiplexer (DEMUX) module. This demultiplexer can receive a single raw audio data stream and distribute and replicate it in real-time and losslessly into multiple parallel audio data streams, which are then sent to the corresponding PCM nodes. The so-called demultiplexer is essentially a data routing switch. It does not change information such as waveform and spectrum in the original audio data. Instead, it copies and distributes a copy of the input data to multiple independent output channels. Each output channel corresponds to an independent audio data stream. In other words, although these multiple audio data streams generated by the demultiplexer are physically separate channels, the audio content they carry is exactly the same as the original input audio data. The number of multiple audio data streams can be dynamically configured according to the number of audio processing uses currently actually activated by the target vehicle. The value range is from 2 to the total number of currently activated audio processing uses.

[0039] The aforementioned target form refers to a communication protocol that supports the multiplexing and transmission of multiplexed audio signals through a single physical transmission link. Specifically, it can be any one or a combination of the following: such as... Figure 3 The integrated circuit shown has a built-in audio bus (Inter-ICSound Bus, I) 2 The S protocol and its evolution protocols, the Serial Low-power Inter-chip Media Bus (SLIMbus) protocol, digital broadcast audio protocols such as S / PDIF (Sony / Philips Digital Interface Format) or AES3 (Audio Engineering Society / European Broadcasting Union 3), the Automotive Audio Bus (A2B) protocol, and other communication protocols that follow similar principles are not described in detail in this specification.

[0040] In summary, multi-path distribution achieved through demultiplexers has advantages such as low latency, low resource consumption, and high data consistency, making it particularly suitable for the real-time and reliability requirements of vehicle environments.

[0041] Of course, besides serving as a built-in hardware unit of the DSP chip to simplify common audio operations such as data splitting and address decoding, the aforementioned demultiplexer can also enable the DSP to act as a main controller, utilizing interfaces such as General Purpose Input / Output (GPIO), Serial Peripheral Interface (SPI), and Integrated Circuit Bus (I2C). 2 The C interface can control an external independent demultiplexer to achieve the same function, and this specification does not limit this.

[0042] In addition, such as Figure 3 As shown, the DSP can also be configured with a multiplexer (MUX) corresponding to the demultiplexer (DEMUX). This multiplexer is primarily used to preprocess and integrate audio data from different sources before the demultiplexer performs multiplexing operations. Specifically, before the demultiplexer generates multiple audio data streams from the original audio data, the multiplexer can perform standardized processing on the following two types of data, targeting parameters such as uniform sampling rate, bit depth, and number of channels, and then concatenate or multiplex them into a logically coherent, standardized data stream:

[0043] 1. Recording data from raw audio data collected by voice input devices: i.e., ambient sounds or human voices that need to be collected, analyzed and processed.

[0044] 2. The playback data is not collected by the voice input device, but actively output by the target vehicle: for example, the music being played by the in-vehicle entertainment system, navigation prompts, system prompts, or reverse sound wave signals generated by the active noise cancellation system.

[0045] The following is based on Figure 4Taking the hardware layer and hardware abstraction layer as an example, this paper introduces an audio processing system that includes a voice input device and an audio processing device. Assuming the voice input device is a vehicle microphone, the raw audio data it collects includes two types of audio content: user voice questions for voice assistant interaction, such as "Hi Xiao X, what's the weather like tomorrow?", and song audio for in-vehicle entertainment system audio processing. This audio content is then input into the DSP chip via TDM. In the DSP, a multiplexer combines the recording data from the raw audio data with the media data currently playing in the target vehicle, such as the background music of the song audio. A demultiplexer then distributes the raw audio data into two audio data streams with identical audio content. These two audio data streams can be written to the speech recognition recording PCM node and the KTV recording PCM node respectively through the DSP driver of the corresponding DSP chip in the kernel layer. These two nodes, the speech recognition recording PCM node and the KTV recording PCM node, are involved in audio processing, thus establishing a one-to-one correspondence between the audio data streams and the PCM nodes.

[0046] Step S206: Read the audio data stream from the target PCM node through the target audio processing process, and execute the audio processing operation corresponding to the binding purpose. The target audio processing process is matched with the audio processing purpose of the target PCM node.

[0047] Multiple independent audio processing processes can run within a vehicle's audio processing equipment. Each process is bound to one or more specific audio processing purposes upon startup. These processes actively listen for and read the corresponding audio data streams from the PCM nodes that match their bound purpose. Once the data stream is acquired, each process independently and specifically executes the specific audio processing operation corresponding to its bound purpose, such as speech recognition and parsing, call noise reduction, and KTV song sound effect processing, thereby completing the entire audio processing flow. Furthermore, users can control and manage these audio processing operations through the in-vehicle application corresponding to the audio processing process.

[0048] After completing the targeted operations of each audio processing step, the system can further integrate and play the audio output. Specifically, it can mix the resulting audio data stream with other audio data streams actively output by the target vehicle in real time, and then drive the vehicle's audio speakers (Audio Output Device, AUD) to play the mixed audio data stream as the final audio signal. Each audio processing step generates a resulting audio data stream corresponding to its intended purpose after executing its bound audio processing operation. For example, the voice assistant process generates a voice response data stream after parsing user commands, the entertainment system process generates a KTV music playback data stream, and the active noise cancellation process generates an inverted sound wave data stream. Furthermore, the mixing can be considered as the mixer corresponding to the target vehicle receiving and mixing the resulting audio data stream with other audio data streams; the resulting audio data stream can be determined based on the execution results of each audio processing operation; the resulting audio data stream is playback data independent of the audio processing flow, such as navigation prompts, system alarm sounds, and Bluetooth phone voice messages.

[0049] Those skilled in the art will understand that the above mixing operations can be implemented in different ways depending on the actual scenario. For example, the hardware resources of the DSP chip can be reused to complete the mixing operation within the DSP's built-in mixing module, or the data can be transmitted to an external mixing chip for processing via an A2B or other automotive audio bus. This external mixing chip can be further configured in devices such as audio speakers to form a slot layout for managing different audio channels, or it can be configured in an independent mixing device for the same purpose. Alternatively, software-level mixing can be implemented within an operating system audio framework such as Android Automotive HAL. The so-called slot layout refers to the implementation of multi-device, multi-scenario audio management through logical abstraction and dynamic routing, such as... Figure 5 As shown, assuming that the Slot layout in an AUD is configured with Slot nodes corresponding to channels A1 to A9, i.e. Figure 5 Slots 0 through 8 (or links) are assigned to different audio sources. For example, media uses A1, A2, A3, and A4 in different locations within the cabin, such as the front left channel, front right channel, rear left channel, and rear right channel. Voice recognition outputs through A5, alarm audio outputs separately through A6, karaoke audio outputs through A8, navigation outputs through A9, etc. This manual does not limit this; at the same time, for example... Figure 5 Slots 9 through 8 are in an idle state and can be configured with other audio sources.

[0050] The above mixing operations achieve dynamic audio mixing through the following two parallel mechanisms, which can work independently or in combination:

[0051] In the vehicle state perception mechanism, the aforementioned system can collect multi-dimensional current operating state parameters of the target vehicle in real time, including but not limited to the vehicle's own state such as speed, gear, and driving mode; the state of the passengers in the cabin such as driver fatigue index and occupant position distribution; and the external environmental state such as road type. Based on this, the system dynamically adjusts the corresponding volume gain: for example, when the vehicle speed exceeds 80 km / h, the system automatically increases the output gain of safety alarm audio streams by 40%; if driver fatigue is detected, the mixing weight of warning sound effects is significantly increased to 70%. Assuming the aforementioned audio stream is considered the resulting audio data stream, the volume of other audio data streams can be reduced based on the aforementioned volume gain to highlight the resulting audio data stream in the mix.

[0052] Furthermore, in the mixing priority mechanism, each audio data stream to be mixed, as well as other audio data streams, can be predefined with a priority value of 0-100, which is strictly positively correlated with the output volume gain. Preferably, priority determination can follow a hierarchical rule: for safety alarms, assuming a priority of 90-100, lower priority streams can be forcibly interrupted; for voice interaction commands, assuming a priority of 80-89, a volume reduction strategy is automatically activated upon triggering, applying a 50% gain attenuation to competing media / KTV audio; for navigation prompts, assuming a priority of 70-79, the volume is automatically reduced by 30% when coexisting with non-emergency voice prompts. The above two mechanisms can work together through conflict resolution rules: for example, when state awareness and priority strategies conflict, such as entertainment audio during high-speed driving, the constraint on the safety state can have the highest arbitration authority by default; at the same time, the priority mechanism constructs the framework of the basic mixing system, and the state parameters are dynamically corrected within this framework. Taking a KTV scenario as an example: when a voice wake-up event occurs, the priority mechanism activates the Duck strategy to attenuate the KTV stream gain by 50%. On the other hand, when the vehicle status perception module detects that the vehicle speed is >60km / h, it can add 20% gain compensation to other audio data streams used to announce the current vehicle speed status.

[0053] The following is based on Figure 4Taking the application layer and hardware abstraction layer as examples, the above audio processing system will be further introduced. As mentioned earlier, after the speech recognition recording PCM node and the KTV recording PCM node are respectively written with two audio data streams of the same audio content, for the speech recognition recording PCM node, its audio data stream can be read by the corresponding speech recognition process, and then handed over to the speech recognition APP in the application layer to notify the audio service process to perform specific audio processing. Subsequently, the processing result is written to the media playback KTV node and then transmitted to the DSP chip through the DSP driver. At the same time, for the KTV recording PCM node, its audio data stream can be read by the corresponding microphoneless KTV process. Based on the microphoneless KTV APP control, the singing and music data are transmitted to the DSP chip through the DSP driver. At this point, the two audio data streams after performing audio processing operations have been sent to the DSP chip, which can input the two audio data streams to the audio speakers for mixing and playback. Between the application layer and the hardware abstraction layer, there is actually a logical framework (FWK) layer. This FWK layer not only provides a tracking mechanism for application layer apps, allowing each app to uniquely identify a target process through its process ID (PID) and thus capture its internal data, but also... Figure 4 As shown by the dotted arrow, when a specific process, such as voice recognition, is triggered, the chip controller instructs the DSP chip to notify the audio speaker to perform Duck processing on the preset mixing link used for KTV playback. Of course, audio from other media apps is also usually processed at the same time, the purpose of which is to ensure the accuracy of voice commands and the interactive experience. Naturally, when a dialogue, such as voice recognition, ends, the FWK layer can also be triggered to notify the audio speaker to cancel the Duck processing on the preset mixing link used for KTV playback, thus restoring the original volume playback.

[0054] After performing each audio processing operation, this manual can also achieve sound field spatial separation through a dynamic routing strategy, thereby solving the signal crosstalk problem of multiple output audio streams.

[0055] In one embodiment, the system can isolate the interfering audio data streams from the resulting audio data stream and other audio data streams actively output by the target vehicle and input them to different audio speakers in the target vehicle. The resulting audio data stream is determined based on the execution results of each audio processing operation. The criteria for determining whether interference has occurred can be configured by the user or based on preset conflict relationships between different audio data streams. For example, when the target vehicle simultaneously activates the navigation function and the microphoneless KTV entertainment function while driving, a driving mode activation command is automatically sent to the audio routing controller. This command can instruct other audio data streams corresponding to the navigation prompts to be redirected from the vehicle-wide universal audio speaker routing to the dedicated speaker in the driver's seat headrest, while the resulting audio data stream used for audio processing of the in-vehicle entertainment system remains output to the aforementioned audio speakers. This avoids the following potential problems: the navigation prompts with higher mixing priority periodically triggering the Duck strategy, resulting in forced reduction of the audio stream gain of the media or KTV on the in-vehicle speakers, and frequent interruptions of KTV singing effects, disrupting the continuous experience of entertainment audio.

[0056] Figure 6 This is a schematic structural diagram of an electronic device according to an exemplary embodiment. Please refer to... Figure 6 At the hardware level, the electronic device includes a processor 602, an internal bus 610, a network interface 604, memory 606, and non-volatile memory 408, and may also include other necessary hardware. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it, forming a risk code detection device at the logical level. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0057] Figure 7 This invention illustrates a block diagram of a vehicle-based audio processing device according to an embodiment of the invention. Please refer to... Figure 7 The device can be applied to vehicles, which include audio processing equipment configured with multiple pulse code modulation (PCM) nodes and multiple audio processing processes. Each PCM node has a corresponding different audio processing purpose, and each audio processing process is bound to a different audio processing purpose. The device includes:

[0058] The data acquisition unit 702 is used to acquire raw audio data for the target vehicle. The raw audio data includes multiple audio contents, each of which corresponds to a different audio processing purpose in the target vehicle.

[0059] The data stream generation unit 704 is used to generate multiple audio data streams based on the original audio data, and write each audio data stream to a different PCM node. The audio content of each audio data stream is the same as the original audio data.

[0060] The report content sending unit 706 is used to read the audio data stream from the target PCM node through the target audio processing process and execute the audio processing operation corresponding to the binding purpose, wherein the target audio processing process is matched with the audio processing purpose of the target PCM node.

[0061] Optionally, the audio processing device further includes a digital signal processor (DSP), which includes a demultiplexer; the data stream generation unit 704 is specifically used for:

[0062] The raw audio data is input into the DSP in a preset target format, and the raw audio data is distributed into the multi-channel audio data stream based on the demultiplexer.

[0063] Optionally, the target form can be any of the following:

[0064] Integrated circuit built-in audio bus protocols and their evolution protocols, serial low-power chip-to-chip media bus protocols, digital broadcast audio protocols, automotive audio bus protocols, and other communication protocols that support multiplexing and transmitting multiple audio signals through a single physical transmission link.

[0065] Optionally, the device further includes:

[0066] The mixing unit is used to mix the resulting audio data stream and other audio data streams actively output by the target vehicle, and play the mixed audio data stream. The resulting audio data stream is determined according to the execution results corresponding to each audio processing operation.

[0067] Optionally, each resulting audio data stream and other audio data streams have a corresponding mixing priority; the mixing unit is specifically used for:

[0068] The volume gain of each audio data stream to be mixed and the other audio data streams are set according to the current operating status parameters of the target vehicle; and / or,

[0069] The volume gain of each audio data stream to be mixed and other audio data streams is set according to the mixing priority, and the volume gain is positively correlated with the mixing priority of the corresponding audio content.

[0070] Optionally, the mixing unit is specifically used for:

[0071] The corresponding volume gain is determined based on the current operating status parameters.

[0072] The volume of the other audio data streams is reduced based on the volume gain.

[0073] Optionally, the device further includes:

[0074] An audio isolation input unit is used to isolate and input the resulting audio data stream and other audio data streams actively output by the target vehicle that interfere with each other to different audio speakers of the target vehicle. The resulting audio data stream is determined according to the execution result corresponding to each audio processing operation.

[0075] Optionally, the audio processing uses include at least the following two:

[0076] Voice assistant interaction processing, in-vehicle entertainment system audio processing, in-vehicle call processing, voiceprint-based biometric recognition, driver state analysis based on audio features, recording function processing, occupant emotion recognition, in-vehicle active noise reduction processing, and cabin sound field optimization processing.

[0077] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0078] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0079] Based on the same concept as the above method, this specification also provides an audio processing system applied to a vehicle. The vehicle includes an audio processing device configured with multiple pulse code modulation (PCM) nodes and multiple audio processing processes. Each PCM node has a corresponding different audio processing purpose, and each audio processing process is bound to a different audio processing purpose; wherein:

[0080] The audio processing device is used to acquire raw audio data for a target vehicle, the raw audio data including multiple audio contents, each audio content corresponding to a different audio processing purpose in the target vehicle; and to generate multiple audio data streams based on the raw audio data, and write each audio data stream to a different PCM node, the audio content of each audio data stream being the same as the raw audio data.

[0081] The audio processing process is used to read audio data streams from a target PCM node whose audio processing purpose matches that of the target PCM node, and to perform audio processing operations corresponding to that binding purpose.

[0082] Optionally, the audio processing device further includes a digital signal processor (DSP), which includes a demultiplexer;

[0083] The audio processing device is specifically used for:

[0084] The raw audio data is input into the DSP in a preset target format, and the raw audio data is distributed into the multi-channel audio data stream based on the demultiplexer.

[0085] Optionally, the target form may be any one or more of the following:

[0086] Integrated circuit built-in audio bus protocols and their evolution protocols, serial low-power chip-to-chip media bus protocols, digital broadcast audio protocols, automotive audio bus protocols, and other communication protocols that support multiplexing and transmitting multiple audio signals through a single physical transmission link.

[0087] Optionally, the audio processing procedure is further used for:

[0088] The resulting audio data stream and other audio data streams actively output by the target vehicle are mixed, and the mixed audio data stream is played. The resulting audio data stream is determined based on the execution results corresponding to each audio processing operation.

[0089] Optionally, each resulting audio data stream and other audio data streams have a corresponding mixing priority; the audio processing process is specifically used for:

[0090] The volume gain of each audio data stream to be mixed and the other audio data streams are set according to the current operating status parameters of the target vehicle; and / or,

[0091] The volume gain of each audio data stream to be mixed and other audio data streams are set according to the mixing priority, and the volume gain is positively correlated with the mixing priority of the corresponding audio content.

[0092] Optionally, the audio processing process is specifically used for:

[0093] The corresponding volume gain is determined based on the current operating status parameters.

[0094] The volume of the other audio data streams is reduced based on the volume gain.

[0095] Optionally, the audio processing procedure is further used for:

[0096] The resulting audio data stream and other audio data streams actively output by the target vehicle that interfere with each other are isolated and input to different audio speakers of the target vehicle. The resulting audio data stream is determined according to the execution result corresponding to each audio processing operation.

[0097] Optionally, the audio processing uses include at least the following two:

[0098] Voice assistant interaction processing, in-vehicle entertainment system audio processing, in-vehicle call processing, voiceprint-based biometric recognition, driver state analysis based on audio features, recording function processing, occupant emotion recognition, in-vehicle active noise reduction processing, and cabin sound field optimization processing.

[0099] Based on the same concept as the methods described above, this specification also provides a vehicle, including: a processor; a memory for storing processor-executable instructions; and a voice input device; wherein the processor executes the executable instructions to implement the steps of the method as described in any of the above embodiments.

[0100] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0101] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0102] The embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0103] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0104] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0105] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0106] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0107] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0108] Therefore, specific embodiments of the subject matter have been described. Furthermore, the processes depicted in the figures are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0109] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of protection of this specification.

Claims

1. An audio processing method, characterized in that, Applied to vehicles, the vehicles include an audio processing device configured with multiple pulse code modulation (PCM) nodes and multiple audio processing processes, each PCM node having a corresponding different audio processing purpose, and each audio processing process being bound to a different audio processing purpose; the method includes: Acquire raw audio data for the target vehicle, the raw audio data including multiple audio contents, each audio content corresponding to a different audio processing purpose in the target vehicle; Multiple audio data streams are generated based on the original audio data, and each audio data stream is written to a different PCM node. The audio content of each audio data stream is the same as the original audio data. The target audio processing process reads the audio data stream from the target PCM node and performs the audio processing operation corresponding to the binding purpose, wherein the target audio processing process is matched with the audio processing purpose of the target PCM node.

2. The method according to claim 1, characterized in that, The audio processing device further includes a digital signal processor (DSP), which includes a demultiplexer; the step of generating multiple audio data streams based on the original audio data includes: The raw audio data is input into the DSP in a preset target format, and the raw audio data is distributed into the multi-channel audio data stream based on the demultiplexer.

3. The method according to claim 2, characterized in that, The target form is any one or more of the following: Integrated circuit built-in audio bus protocols and their evolution protocols, serial low-power chip-to-chip media bus protocols, digital broadcast audio protocols, automotive audio bus protocols, and other communication protocols that support multiplexing and transmitting multiple audio signals through a single physical transmission link.

4. The method according to claim 1, characterized in that, The method further includes: The resulting audio data stream and other audio data streams actively output by the target vehicle are mixed, and the mixed audio data stream is played. The resulting audio data stream is determined according to the execution results corresponding to each audio processing operation.

5. The method according to claim 4, characterized in that, Each resulting audio data stream and other audio data streams have a corresponding mixing priority; mixing the resulting audio data stream and the other audio data streams includes: The volume gain of each audio data stream to be mixed and the other audio data streams are set according to the current operating status parameters of the target vehicle; and / or, The volume gain of each audio data stream to be mixed and other audio data streams is set according to the mixing priority, and the volume gain is positively correlated with the mixing priority of the corresponding audio content.

6. The method according to claim 5, characterized in that, The step of setting the volume gain of each audio data stream to be mixed and the other audio data streams according to the current operating status parameters of the target vehicle includes: The corresponding volume gain is determined based on the current operating status parameters. The volume of the other audio data streams is reduced based on the volume gain.

7. The method according to claim 1, characterized in that, The method further includes: The resulting audio data stream and other audio data streams actively output by the target vehicle that interfere with each other are isolated and input to different audio speakers of the target vehicle. The resulting audio data stream is determined according to the execution result corresponding to each audio processing operation.

8. The method according to any one of claims 1 to 7, characterized in that, The audio processing uses include at least the following two: Voice assistant interaction processing, in-vehicle entertainment system audio processing, in-vehicle call processing, voiceprint-based biometric recognition, driver state analysis based on audio features, recording function processing, occupant emotion recognition, in-vehicle active noise reduction processing, and cabin sound field optimization processing.

9. An audio processing system, characterized in that, Applied to vehicles, the vehicle includes an audio processing device configured with multiple pulse code modulation (PCM) nodes and multiple audio processing processes. Each PCM node has a corresponding different audio processing purpose, and each audio processing process is bound to a different audio processing purpose; wherein: The audio processing device is used to acquire raw audio data for a target vehicle, the raw audio data including multiple audio contents, each audio content corresponding to a different audio processing purpose in the target vehicle; and to generate multiple audio data streams based on the raw audio data, and write each audio data stream to a different PCM node, the audio content of each audio data stream being the same as the raw audio data. The audio processing process is used to read audio data streams from target PCM nodes that match the audio processing purpose, and to perform audio processing operations corresponding to the binding purpose.

10. The system according to claim 9, characterized in that, The audio processing device also includes a digital signal processor (DSP), and the DSP includes a demultiplexer; The audio processing device is specifically used for: The raw audio data is input into the DSP in a preset target format, and the raw audio data is distributed into the multi-channel audio data stream based on the demultiplexer.

11. The system according to claim 10, characterized in that, The target form is any one or more of the following: Integrated circuit built-in audio bus protocols and their evolution protocols, serial low-power chip-to-chip media bus protocols, digital broadcast audio protocols, automotive audio bus protocols, and other communication protocols that support multiplexing and transmitting multiple audio signals through a single physical transmission link.

12. The system according to claim 9, characterized in that, The audio processing process is also used for: The resulting audio data stream and other audio data streams actively output by the target vehicle are mixed, and the mixed audio data stream is played. The resulting audio data stream is determined according to the execution results corresponding to each audio processing operation.

13. The system according to claim 12, characterized in that, Each resulting audio data stream and other audio data streams have a corresponding mixing priority; the audio processing process is specifically used for: The volume gain of each audio data stream to be mixed and the other audio data streams are set according to the current operating status parameters of the target vehicle; and / or, The volume gain of each audio data stream to be mixed and other audio data streams is set according to the mixing priority, and the volume gain is positively correlated with the mixing priority of the corresponding audio content.

14. The system according to claim 13, characterized in that, The audio processing process is specifically used for: The corresponding volume gain is determined based on the current operating status parameters. The volume of the other audio data streams is reduced based on the volume gain.

15. The system according to claim 9, characterized in that, The audio processing process is also used for: The resulting audio data stream and other audio data streams actively output by the target vehicle that interfere with each other are isolated and input to different audio speakers of the target vehicle. The resulting audio data stream is determined according to the execution result corresponding to each audio processing operation.

16. The system according to any one of claims 9 to 15, characterized in that, The audio processing uses include at least the following two: Voice assistant interaction processing, in-vehicle entertainment system audio processing, in-vehicle call processing, voiceprint-based biometric recognition, driver state analysis based on audio features, recording function processing, occupant emotion recognition, in-vehicle active noise reduction processing, and cabin sound field optimization processing.

17. A vehicle, characterized in that, include: processor; A memory for storing processor-executable instructions and a voice input device; wherein the processor implements the steps of the method as described in any one of claims 1-8 by running the executable instructions.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 8.

19. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-8.

Images