Vehicle audio control method, device, apparatus and medium

Abstract

The application discloses a vehicle-mounted audio control method, device and equipment and a medium, and belongs to the technical field of vehicle control. The vehicle-mounted audio control method comprises the following steps: calling a large language model after training to analyze a feature representation sequence representing a vehicle state and a personnel state in a cabin, so as to obtain a control action sequence for a vehicle-mounted audio system; according to the operation types of each audio control action in the control action sequence, distributing each audio control action in the control action sequence to a standard data channel and an atomized control channel; processing the audio control actions of the corresponding operation types through the standard data channel and the atomized control channel, so as to output an audio data stream and an atomized parameter instruction for the vehicle-mounted audio system; and changing the audio effect in the cabin according to the audio data stream and the atomized parameter instruction. The application solves the technical problems of low intelligent degree, slow response to cabin scene changes and insufficient control precision of related technologies.

Description

Technical Field

[0001] This invention belongs to the field of vehicle control technology, and particularly relates to an in-vehicle audio control method, device, equipment and medium. Background Technology

[0002] As smart cockpits evolve towards an Artificial Intelligence Operating System (AIOS), in-vehicle audio systems have transformed from simple entertainment playback tools into complex, immersive interactive systems. In the relevant Android Automotive OS architecture (Android Automotive OS is a complete Android operating system that runs directly on automotive hardware and is deeply customized for the in-vehicle environment, responsible for managing the vehicle's media, navigation, communication, and other infotainment functions), audio management primarily relies on the AudioFlinger and AudioPolicyService services for policy routing. In the audio processing architecture of related technologies, various audio processing policies (such as interrupting music during navigation and muting when reversing) are typically hardcoded into static configuration files during the compilation phase. Because audio processing depends on static configuration files, the in-vehicle audio system can only switch between a few preset modes. This results in a rigid audio control strategy and low intelligence when facing complex and ever-changing cockpit scenarios, requiring a fixed audio processing strategy to respond. Furthermore, the audio processing architecture in related technologies can only adjust limited parameters (such as volume) through standard application programming interfaces (APIs). The audio processing chain is lengthy and rigid, unable to dynamically reconstruct the underlying mixing logic and post-processing topology in real time, resulting in sluggish response to changes in the cabin environment. Upper-layer applications (APPs) cannot directly access the advanced processing capabilities of the underlying DSP (Digital Signal Processor), and cannot invoke specific noise reduction algorithms or fine-grained routing control. Existing in-vehicle audio systems lack a deep semantic understanding of vehicle and user states, which cannot be translated into precise adjustments of underlying acoustic parameters, resulting in insufficient control precision. Summary of the Invention

[0003] This invention provides an in-vehicle audio control method, device, equipment, and medium to address the technical problems of low intelligence, slow response to changes in the cabin environment, and insufficient control precision in related technologies.

[0004] A first aspect of the present invention provides an in-vehicle audio control method, comprising: acquiring a feature representation sequence characterizing a vehicle state and the state of occupants in the cabin; parsing the feature representation sequence by invoking a trained large language model to obtain a control action sequence for an in-vehicle audio system, the control action sequence including multiple audio control actions; distributing each audio control action in the control action sequence to a parallel standard data channel and an atomic control channel according to the operation type of each audio control action in the control action sequence; processing the audio control actions of the corresponding operation type through the standard data channel and the atomic control channel to output an audio data stream and atomic parameter instructions for the in-vehicle audio system; and changing the audio processing topology of the in-vehicle audio system according to the audio data stream and the atomic parameter instructions to change the audio effect in the cabin.

[0005] In conjunction with the first aspect, in some embodiments, obtaining the feature representation sequence characterizing the vehicle state and the state of the occupants in the cabin includes: obtaining vehicle state data and cabin environment data characterizing the state of the occupants in the cabin, wherein the vehicle state data includes vehicle body state parameters and status information of the in-vehicle audio application, and the cabin environment data includes at least one of the occupants' voice commands, position, and posture; constructing structured prompt words based on the vehicle state data and the cabin environment data; and converting the structured prompt words into the feature representation sequence.

[0006] In conjunction with the first aspect, in some embodiments, acquiring vehicle status data and cabin environment data characterizing the status of occupants includes: collecting voice commands from occupants via a microphone; capturing cabin images via a camera and identifying the position and / or posture of each occupant based on the cabin images; acquiring at least one of the following vehicle status parameters via a CAN bus and / or Ethernet: vehicle speed, gear position, window status, and environmental noise floor data; and acquiring status information of various in-vehicle audio applications running on the vehicle's infotainment system.

[0007] In conjunction with the first aspect, in some embodiments, the step of calling the trained large language model to parse the feature representation sequence to obtain a control action sequence for the in-vehicle audio system includes: calling the large language model to perform semantic analysis and reasoning on the feature representation sequence to determine the current user intent of each person in the cabin or the target person; and calling the large language model to process the current user intent to obtain the control action sequence.

[0008] In conjunction with the first aspect, in some embodiments, the operation types of the audio control actions include streaming media start / stop actions and acoustic parameter adjustment actions; the step of distributing each audio control action in the control action sequence to a parallel standard data channel and an atomic control channel according to the operation type of each audio control action in the control action sequence includes: distributing the streaming media start / stop actions in the control action sequence to the standard data channel; and distributing the acoustic parameter adjustment actions in the control action sequence to the atomic control channel.

[0009] In conjunction with the first aspect, in some embodiments, the step of processing audio control actions of corresponding operation types through the standard data channel and the atomic control channel to output audio data streams and atomic parameter instructions for the vehicle audio system includes: for streaming media start / stop actions in the control action sequence, responding to the streaming media start / stop actions through the standard audio processing link within the standard data channel to control the transmission of audio data streams; for each acoustic parameter adjustment action in the control action sequence, processing the acoustic parameter adjustment action through the corresponding atomic service within the atomic control channel to obtain the atomic parameter instructions corresponding to the audio control action.

[0010] In conjunction with the first aspect, in some embodiments, the acoustic parameter adjustment class includes multiple adjustment subclasses, and the atomic control channel includes multiple atomic services corresponding to the multiple adjustment subclasses; the step of processing the acoustic parameter adjustment action through the corresponding atomic service in the atomic control channel to obtain the atomic parameter instruction corresponding to the audio control action includes: calling the atomic service in the atomic control channel corresponding to the adjustment subclass of the acoustic parameter adjustment action, mapping the acoustic parameter adjustment action to hardware control parameters and the node identifier of the audio processing node, and generating the atomic parameter instruction corresponding to the acoustic parameter adjustment action based on the hardware control parameters and the node identifier.

[0011] In conjunction with the first aspect, in some embodiments, the atomic service generates an atomic parameter instruction corresponding to the acoustic parameter adjustment action based on the hardware control parameters and the node identifier, including: querying the current audio processing topology of the vehicle audio system; for the acoustic parameter adjustment action in the control action sequence, generating a modification instruction for the target node through the atomic service corresponding to the adjustment subclass of the acoustic parameter adjustment action based on the current audio processing topology, the hardware control parameters corresponding to the acoustic parameter adjustment action, and the node identifier, and using the modification instruction as the atomic parameter instruction corresponding to the acoustic parameter adjustment action.

[0012] In conjunction with the first aspect, in some embodiments, the step of changing the audio processing topology of the in-vehicle audio system according to the audio data stream and the atomication parameter instructions to change the audio effect in the cabin includes: encapsulating the atomication parameter instructions output by the atomication control channel into a remote procedure call request; modifying the state of the target node of the in-vehicle audio system according to the remote procedure call request to reconstruct the audio processing topology in the cabin, so as to improve the audio effect in the vehicle cabin.

[0013] In conjunction with the first aspect, in some embodiments, the audio control method is applied to an in-vehicle infotainment system. The audio processing architecture of the in-vehicle infotainment system includes a multimodal perception layer, a decision layer, a middleware layer, a hardware abstraction layer, and a hardware processor layer arranged from top to bottom. The standard data channel and the atomic control channel are located in the middleware layer.

[0014] In conjunction with the first aspect, in some embodiments, the multimodal perception layer acquires a feature representation sequence characterizing the vehicle state and the state of the occupants in the cabin; the decision layer calls a trained large language model to parse the feature representation sequence to obtain a control action sequence for the in-vehicle audio system, the control action sequence including multiple audio control actions; the middleware layer distributes each audio control action in the control action sequence to parallel standard data channels and atomic control channels according to the operation type of each audio control action in the control action sequence, and processes the audio control actions of the corresponding operation type through the standard data channels and the atomic control channels to output an audio data stream and atomic parameter instructions for the in-vehicle audio system; based on the communication between the hardware abstraction layer and the hardware processor layer, the audio processing topology of the in-vehicle audio system is changed according to the audio data stream and the atomic parameter instructions to change the audio effect in the cabin.

[0015] In conjunction with the first aspect, in some embodiments, the hardware abstraction layer includes a first interface and a second interface. The step of altering the audio processing topology of the in-vehicle audio system based on the audio data stream and the atomic parameter instructions, according to the communication between the hardware abstraction layer and the hardware processor layer, to change the audio effect in the cabin, includes: receiving atomic parameter instructions transmitted via socket communication by the atomic service in the atomic control channel through the first interface of the hardware abstraction layer, and encapsulating the atomic parameter instructions into a remote procedure call request; receiving the audio data stream output by the standard data channel by the second interface of the hardware abstraction layer; responding to the remote procedure call request by the audio processing engine running in the hardware processor layer, locating the corresponding node in the audio processing topology of the audio processing engine; and modifying the value of the control buffer corresponding to the node to change the processing logic of the audio digital signal processor in the hardware processor layer, thereby reconstructing the audio processing topology.

[0016] In a second aspect of the invention, an audio control device for a vehicle cabin is provided, comprising: a data acquisition unit for acquiring a feature representation sequence characterizing the vehicle state and the environment inside the vehicle cabin; an intent parsing unit for invoking a trained large language model to parse the feature representation sequence to obtain a control action sequence for an in-vehicle audio system, the control action sequence including multiple audio control actions; a data processing unit for distributing each audio control action in the control action sequence to a parallel standard data channel and an atomic control channel according to the operation type of each audio control action in the control action sequence, processing the audio control actions of corresponding operation types through the standard data channel and the atomic control channel to output an audio data stream and atomic parameter instructions for the in-vehicle audio system; and a topology changing unit for changing the audio processing topology of the in-vehicle audio system according to the audio data stream and the atomic parameter instructions to change the audio effect inside the cabin.

[0017] In a third aspect of the invention, an electronic device is provided, including one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, the at least one piece of program code being loaded and executed by the one or more processors to implement the in-vehicle audio control method according to any embodiment of the first aspect.

[0018] In a fourth aspect of the invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the in-vehicle audio control method described in any embodiment of the first aspect.

[0019] The one or more technical solutions provided in the embodiments of the present invention achieve at least the following technical effects or advantages: By acquiring feature representation sequences characterizing the vehicle state and the state of the occupants in the cabin, and then parsing these feature representation sequences using a trained large language model, a control action sequence for the in-vehicle audio system is obtained. This control action sequence includes multiple audio control actions. Based on the operation type of each audio control action in the control action sequence, these actions are distributed to parallel standard data channels and atomic control channels. The standard data channels and atomic control channels process the corresponding operation types of the audio control actions to output an audio data stream and atomic parameter instructions for the in-vehicle audio system. The audio processing topology of the in-vehicle audio system is modified according to the audio data stream and atomic parameter instructions, thereby changing the audio effect in the cabin. This technical solution generates audio control sequences based on the perception of multimodal information and the semantic understanding of a large language model, realizing the mapping from multimodal context to multiple audio control actions. Furthermore, by constructing a combination of atomic control channels parallel to the standard data channel and the audio processing topology, the execution of audio control actions bypasses the cumbersome and lengthy audio processing chain, improving the response speed of the underlying hardware resources to the audio control actions.

[0020] Based on the implementation methods provided in the above aspects, the present invention can be further combined to provide more implementation methods. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0022] Figure 1 A flowchart of an in-vehicle audio control method according to some embodiments of the present invention is shown; Figure 2 A system architecture diagram of an in-vehicle audio system according to some embodiments of the in-vehicle audio control method of the present invention is shown; Figure 3 A schematic diagram of the structure of an in-vehicle audio control device according to some embodiments of the present invention is shown; Figure 4 A schematic diagram of the structure of an electronic device according to some embodiments of the present invention is shown. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] In the embodiments of this specification, the term "multiple" means "two or more", that is, including two or more cases; the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone.

[0025] This invention provides a vehicle audio control method. Figure 1 A flowchart of an in-vehicle audio control method according to some embodiments of the present invention is shown. For example... Figure 1 As shown, the vehicle audio control method provided in this embodiment of the invention includes the following steps S101 to S104.

[0026] Step S101: Obtain the feature representation sequence that characterizes the vehicle state and the state of the occupants in the cabin.

[0027] In some embodiments, obtaining a feature representation sequence characterizing the vehicle state and the state of the occupants in the cabin may include: obtaining vehicle state data and cabin environment data characterizing the state of the occupants in the cabin, wherein the vehicle state data includes vehicle body state parameters and status information of an in-vehicle audio application, and the cabin environment data includes at least one of voice commands, position, and posture of the occupants in the cabin; constructing structured prompt words based on the vehicle state data and the cabin environment data; and converting the structured prompt words into the feature representation sequence.

[0028] It is understood that vehicle body status parameters include at least one of the following: vehicle speed, gear, window status, and ambient noise floor data. Ambient noise floor data refers to the background noise level inside the cabin, which is the sum of all background noise except the target sound source. It represents the level of background noise transmitted from the external environment inside the cabin when the vehicle is stationary or in motion, mainly from engine, tire noise, wind noise, road noise, and air conditioning system, etc.

[0029] In some embodiments, acquiring vehicle status data and cabin environment data characterizing the status of occupants includes: collecting voice commands from occupants via microphone; capturing cabin images via camera, and identifying the position and / or posture of each occupant based on the cabin images. The posture of occupants includes whether they are making a phone call, resting with their eyes closed, chatting, watching a movie, etc. At least one of the following vehicle status parameters is acquired via CAN (Controller Area Network) bus and / or Ethernet: vehicle speed, gear position, window status, and environmental noise floor data, etc.; and the status information of each in-vehicle audio application running on the vehicle's infotainment system is acquired. The status information of the in-vehicle audio applications running on the infotainment system includes the focus status of each currently running in-vehicle audio application, indicating the in-vehicle audio application currently receiving audio focus, such as navigation being broadcast or conferencing software being used for a call.

[0030] In some embodiments, vehicle status data and cabin environment data are assembled according to a preset regular format to generate structured prompt words; the structured prompt words are then segmented and encoded to obtain a feature representation sequence.

[0031] In other embodiments, the data for each modality in the vehicle status data and the cabin environment data representing the status of the occupants in the cabin can be preprocessed first, and the preprocessed standardized data can be assembled according to a preset regularization format to construct structured prompt words.

[0032] Step S102: Call the trained Large Language Model (LLM) to parse the feature representation sequence to obtain the control action sequence for the vehicle audio system, the control action sequence including multiple audio control actions.

[0033] In some embodiments of the present invention, the large language model used is a deep learning model based on the Transformer architecture. Through self-supervised pre-learning on a massive dataset, it learns the general rules and knowledge of language, resulting in a pre-trained basic large language model. Then, using multimodal perception data consisting of vehicle state data and passenger state data as input sources, structured prompts are constructed and converted into feature representation sequences as input samples for the basic large language model. The corresponding user intent and audio control actions are used as supervision labels to fine-tune the basic large language model. This fine-tunes the large language model so that it possesses the ability to autonomously infer the user intent of passengers in the cabin based on the multimodal perception data and generate audio control actions according to the user intent.

[0034] In some embodiments, an artificial intelligence agent running on an NPU (Neural Processing Unit) receives a sequence of feature representations from external input and calls a large language model to parse the sequence of feature representations to obtain a sequence of control actions.

[0035] In some embodiments, invoking the large language model to parse the feature representation sequence to obtain a control action sequence includes: invoking the large language model to perform semantic analysis and reasoning on the feature representation sequence to determine the current user intent of each person in the cabin or the target person; and invoking the large language model to process the current user intent to obtain the control action sequence.

[0036] In other embodiments, the large language model is invoked to parse the feature representation sequence to obtain a control action sequence, including: invoking the large language model to perform semantic analysis and reasoning on the feature representation sequence to determine the current user intent of each person in the cockpit or target person; and determining multiple audio control actions that match the current user intent of each person in the cockpit or target person based on a pre-established mapping relationship between user intent and audio control actions, so as to obtain a control action sequence.

[0037] Understandably, by using a large language model to perform multi-layered semantic understanding, correlation analysis, and logical reasoning on the vehicle state and passenger state carried in the feature representation sequence, the current user intent of each passenger or target passenger can be deeply mined and accurately identified. For example, if the feature representation sequence indicates that a passenger's position is "front passenger" and their posture is "making a phone call," then the large language model identifies the passenger's current user intent as: private call and background noise reduction. For example, if the feature representation sequence indicates that a passenger's position is "front passenger" and their posture is "watching in-car entertainment," then the large language model identifies the passenger's current user intent as: front passenger zone sound field enhancement. For example, if the feature representation sequence indicates that a passenger's position is "rear child seat" and their posture is "sitting quietly," then the large language model identifies the passenger's current user intent as: rear audio limiting and high-frequency harsh sound filtering. For example, if the feature representation sequence carries the position of a person in a cabin as "driver", the posture as "raising hand to issue voice command", the window is open, and the vehicle speed is 100km / h, then the large language model can identify the driver's current user intent as voice pickup focus and surrounding noise shielding.

[0038] Understandably, by performing semantic analysis and reasoning on the feature representation sequence using the large language model, the current user intent of each person in the cabin or the current user intent of a target person can be determined. The target person can be someone in a pre-specified location, such as the driver in the driver's seat or the passenger in the front passenger seat.

[0039] For example, the control action sequence includes multiple audio control actions for the in-vehicle audio system, such as: opening the passenger headrest, lowering the music volume, activating the voice enhancement algorithm, and inserting the navigation voice into the driver's headrest.

[0040] In some embodiments, the control action sequence obtained through the large language model is a semantic control action sequence, which is a set of sequential, procedural control commands in plain text form. Each audio control action is packaged according to a structured data storage format and converted into standardized control instructions. The semantic information of each audio control action can be serialized to convert the audio control action into the form of standardized control instructions. For example, action: open the passenger headrest; target object: passenger seat; type: acoustic parameter adjustment. Another example: action: enhance human voice; target object: rear seat; type: acoustic parameter adjustment. Yet another example: action: adjust volume to 50%; target object: passenger seat; type: acoustic parameter adjustment. Still another example: action: play; target object: passenger seat; type: streaming media start / stop.

[0041] In practice, each audio control action in the control action sequence can be converted into standardized control instructions in protobuf, JSON, or XML data formats.

[0042] Step S103: Based on the operation type of each audio control action in the control action sequence, distribute each audio control action in the control action sequence to a parallel standard data channel and an atomic control channel. Process the audio control actions of the corresponding operation type through the standard data channel and the atomic control channel to output an audio data stream and atomic parameter instructions for the vehicle audio system.

[0043] In some embodiments, audio control actions are categorized into streaming media start / stop actions and acoustic parameter adjustment actions based on their operation type. Streaming media start / stop actions include playing, pausing, stopping, and buffering the audio data stream. Acoustic parameter adjustment actions include volume adjustment, sound effect adjustment, and audio routing adjustment. The streaming media start / stop actions in the control action sequence are distributed to the standard data channel; each acoustic parameter adjustment action in the control action sequence is distributed to the atomized control channel.

[0044] It should be understood that the standard data channel is a channel compatible with the audio data stream transmission of various in-vehicle audio applications, and completes the compliant flow and transmission of audio data streams based on the standard audio architecture. Figure 2 A system architecture diagram of an in-vehicle audio system according to some embodiments of the in-vehicle audio control method of the present invention is shown. For example... Figure 2 As shown, the audio processing chain of the standard audio architecture, from top to bottom, consists of: in-vehicle audio applications, the audio service (AudioService), and the audio mixing service (AudioFlinger). Pulse Code Modulation (PCM) audio data streams generated by in-vehicle audio applications such as music and navigation are transmitted normally through the Audio Service and AudioFlinger. Third-party applications such as music and navigation output PCM audio data streams based on the standard audio interface. The Audio Service and AudioFlinger uniformly manage routing, mixing, and timing adaptation, shielding hardware differences and cockpit audio logic. Third-party in-vehicle audio applications do not need customization or modification, and the in-vehicle audio applications do not need to be aware of the underlying hardware, cockpit architecture, or audio processing algorithms; they only need to output PCM audio data streams according to the standard. The AudioPolicyService and AudioFlinger services are the two basic services of the in-vehicle audio system based on the Android operating system. The AudioPolicyService is the policy maker for the audio system, responsible for policy selection when switching audio devices and volume adjustment policies; the AudioFlinger service is the policy executor for the audio system, responsible for the management of audio streaming devices and the processing and transmission of audio streaming data.

[0045] It should be understood that the atomic control channel and the standard data channel are independent and parallel dual channels in the middleware layer. The atomic control channel does not use the standard audio architecture, but instead communicates directly with the Hardware Abstraction Layer (HAL) through socket communication. The atomic services within the atomic control channel generate atomic parameter instructions that directly reach the HAL through socket communication, bypassing the complex links under the standard audio architecture. For example, in the Android system, atomic parameter instructions skip the complex audio processing architecture such as fwk and native.

[0046] Step S104: Change the audio processing topology of the vehicle audio system according to the audio data stream and the atomization parameter instruction to change the audio effect in the cabin.

[0047] In some embodiments, processing audio control actions of corresponding operation types through the standard data channel and the atomic control channel to output audio data streams and atomic parameter instructions for the vehicle audio system may include: for streaming media start / stop actions in the control action sequence, responding to the streaming media start / stop actions through the standard audio processing link within the standard data channel to control the transmission of audio data streams; for each acoustic parameter adjustment action in the control action sequence, processing the acoustic parameter adjustment action through the corresponding atomic service within the atomic control channel to obtain the atomic parameter instructions corresponding to the audio control action.

[0048] In some embodiments, acoustic parameter adjustment actions can be further subdivided into multiple adjustment subclasses. The complex control strategy for acoustic parameter adjustment is broken down into multiple minimum control units, each of which is an atomic service. Each atomic service implements the acoustic parameter adjustment for its corresponding adjustment subclass. The atomic control channel includes multiple atomic services corresponding to the various adjustment subclasses. Since acoustic parameter adjustment actions are divided into multiple adjustment subclasses such as volume adjustment, audio routing adjustment, and sound effect adjustment, the atomic control channel includes correspondingly configured volume atomic services, routing atomic services, and sound effect atomic services. The volume atomic service adjusts volume through gain adjustment, the routing atomic service changes the audio route by modifying the audio output port, and the sound effect atomic service changes the sound effect by loading or enabling plugins.

[0049] In some embodiments, processing the acoustic parameter adjustment action through the corresponding atomic service within the atomic control channel to obtain the atomic parameter instruction corresponding to the audio control action includes: mapping the acoustic parameter adjustment action to hardware control parameters and the node identifier of the audio processing node through a mapping table; and calling the atomic service corresponding to the adjustment subclass of the acoustic parameter adjustment action within the atomic control channel, so that the atomic service generates the atomic parameter instruction corresponding to the acoustic parameter adjustment action based on the hardware control parameters and the node identifier.

[0050] It should be understood that the audio processing engine includes multiple independently configurable audio processing nodes, each with a node identifier (unique node ID) and a set of adjustable parameters. The mapping table maps audio control actions to the node identifiers and adjustable parameters within the audio processing engine. Hardware control parameters are the parameters that the underlying Audio Digital Signal Processor (ADSP) can execute. Therefore, by looking up the mapping table, semantic acoustic parameter adjustment actions can be mapped to the hardware control parameters that the underlying ADSP can execute.

[0051] In some embodiments, for a volume adjustment action in a control action sequence, the hardware control parameters and node identifier corresponding to the volume adjustment action are obtained by looking up a mapping table. The volume atomic service is then invoked to process the hardware control parameters and node identifier obtained from the table lookup and generate an atomic parameter instruction for adjusting the volume.

[0052] In some embodiments, for the audio routing adjustment action in the control action sequence, the hardware control parameters and node identifier corresponding to the volume routing adjustment action are obtained by looking up the mapping table, and the routing atomic service is called to process the hardware control parameters and node identifier obtained by looking up the table to generate atomic parameter instructions for adjusting the audio routing.

[0053] In some embodiments, for sound effect adjustment actions in the control action sequence, the hardware control parameters and node identifiers corresponding to the sound effect adjustment actions are obtained by looking up a mapping table, and the sound effect atomic service is called to process the hardware control parameters and node identifiers obtained by looking up the table to generate atomic parameter instructions for adjusting sound effects.

[0054] It should be noted that atomic parameter instructions are indivisible, the smallest instruction unit carrying standard control parameters (value, address, mode), performing only a single control action, and executing without interruption or intermediate state.

[0055] In some embodiments, the atomic parameter instructions output by the atomic control channel are encapsulated as a Remote Procedure Call (RPC) request. The state of the target node is modified according to the RPC request, and the audio processing topology of the in-vehicle audio system is reconstructed to change the audio effect in the cabin.

[0056] In some embodiments, generating atomic parameter instructions based on the hardware control parameters and node identifiers includes: querying the current audio processing topology of the vehicle audio system; for acoustic parameter adjustment actions in the control action sequence, generating modification instructions for target nodes based on the current audio processing topology, the hardware control parameters corresponding to the acoustic parameter adjustment actions, and the node identifiers through the atomic service corresponding to the adjustment subclass of the acoustic parameter adjustment actions, and using the modification instructions as the atomic parameter instructions corresponding to the acoustic parameter adjustment actions.

[0057] It should be understood that an audio processing topology is a topological structure of various audio processing nodes arranged according to the direction of audio data flow, processing division, and logical connections. It connects the various audio processing nodes that the audio signal passes through from input, through various algorithms, to speaker output. Each independent functional block is an audio processing node. The audio processing topology is a directed audio data flow topology composed of various audio processing nodes and connections, used to fully define: the nodes and hardware parameter configurations that the audio data flow passes through from the audio source input, algorithm processing, routing and distribution, to the speaker output. Various audio processing nodes include multiple nodes such as audio source input, EQ (equalizer), mixing, vocal enhancement, noise reduction, delay, and power amplifier output.

[0058] In some embodiments, the audio control method provided by this invention is applied to an in-vehicle infotainment system. For example... Figure 2 As shown, the audio processing architecture of the vehicle-mounted system includes, from top to bottom, a multimodal perception layer 10, a decision layer 20, a middleware layer 30, a hardware abstraction layer (HAL) 40, and a hardware processor layer 50. The standard data channel and the atomic control channel are located in the middleware layer 30. Compared to existing technologies that only have a standard data channel in the middleware layer 30, this system employs a dual-channel architecture where the standard data channel and the atomic control channel are configured in parallel.

[0059] The multimodal perception layer 10 serves as the input port for multimodal perception information, responsible for collecting this information. The sensing objects include various sensors such as in-cabin cameras, in-cabin microphones, the Driver Monitoring System (DMS), and the Occupant Monitoring System (OMS). The decision layer 20 incorporates an artificial intelligence agent based on a pre-trained large language model. The middleware layer 30 is the core scheduling and communication hub connecting the upper decision layer 20 and the lower hardware abstraction layer 40, undertaking key functions such as data forwarding, instruction scheduling, channel control, and service decoupling. The hardware abstraction layer 40 is an adaptation and isolation layer that shields the underlying hardware differences and unifies hardware access interfaces. It eliminates interface differences, driver logic, and register specifications between different hardware platforms, chip models, and peripheral devices, providing standardized and unified calling interfaces to the upper-layer middleware and connecting to the drivers and hardware registers of the hardware processor layer 50. The hardware processor layer 50 is the final execution unit, carrying all algorithm calculations, signal processing, instruction execution, and hardware driver work; it is the physical carrier for all logic implementation, signal processing, and device control. This layer contains hardware such as the in-vehicle SoC main control chip, audio DSP, AWE (AudioWeaver) audio processing engine, MCU microcontroller, sensor coprocessor, power amplifier chip, and peripheral interface controller. It is responsible for receiving atomic control commands forwarded by the hardware abstraction layer, running the audio processing topology in real time, collecting raw data from the hardware and providing feedback on the hardware's operating status, and finally completing the closed-loop execution from atomic control commands to physical actions.

[0060] Step S201: The multimodal perception layer 10 acquires a feature representation sequence characterizing the vehicle state and the state of the occupants in the cabin. In some embodiments, the multimodal perception layer performs the following steps S2011-S2013 to acquire a feature representation sequence characterizing the vehicle state and the state of the occupants in the cabin, and outputs the acquired feature representation sequence to the decision layer 20.

[0061] Step S2011, acquire multimodal perception data: Voice commands from occupants are collected via microphone; images of the cabin are captured by an in-cabin camera and processed by an In-Cabin Monitoring System (IMS) to identify the position and / or posture of each occupant, and to monitor the status of the in-vehicle audio application running on the vehicle's infotainment system. At least one of the following data is acquired via CAN bus and / or Ethernet: vehicle speed, gear position, window status, and ambient noise level. The In-Cabin Monitoring System is the visual perception module of the smart cockpit, including a Driver Monitoring System (DMS) to identify the driver's position and posture, and an Occupant Monitoring System (OMS) to identify the occupant's position and posture. For example, the acquired multimodal perception data includes: the passenger in the front passenger seat is in a posture of using a mobile phone to make a call, while the Bluetooth phone app on the vehicle's infotainment system triggers an audio focus request, and the vehicle speed is perceived as 100 km / h.

[0062] Step S2012: Standardize the acquired multimodal perception data to construct structured prompt words.

[0063] Step S2013: Convert the structured prompt words into a feature representation sequence and input it into the decision layer 20.

[0064] Step S202: The artificial intelligence agent running by the decision layer 20 calls the large language model to parse the feature representation sequence to obtain the control action sequence.

[0065] Understandably, the decision layer 20 incorporates an artificial intelligence agent, which includes a perception receiver, an intent understanding module, and a policy generation module. The decision layer 20 can run on the vehicle's neural processing unit (NPU). After the perception receiver of the decision layer 20 receives the feature representation sequence from the multimodal perception layer 10, the intent understanding module of the decision layer 20 calls a large language model to parse the feature representation sequence to obtain the current user intent of each occupant or target occupant. The policy generation module of the decision layer 20 generates a semantic sequence of control actions based on the current user intent of each occupant or target occupant.

[0066] For example, the intent understanding module of decision layer 20 calls the large language model running on the NPU to perform reasoning and determines that the user's need in the passenger seat is the passenger seat conference mode. Then the generated semantic control action sequence is: open the passenger seat headrest, lower the music volume, activate the human voice enhancement algorithm, and put the navigation broadcast voice into the driver's seat headrest.

[0067] Since the decision layer 20 does not directly operate the hardware, the audio control actions in the control action sequence output by the decision layer 20 can, but are not limited to, adopt the standardized Protobuf (Protocol Buffer is a data description language developed by Google) instruction form.

[0068] Step S203: The middleware layer 30 distributes each audio control action in the control action sequence to the standard data channel and the atomic control channel running in parallel in the middleware layer 30 according to the action type of each audio control action in the control action sequence. The audio control actions of the corresponding operation type are processed through the standard data channel and the atomic control channel to output audio data stream and atomic parameter instructions for the vehicle audio system.

[0069] Understandably, the middleware layer 30 acts as a bridge connecting the AI ​​thinking of the decision layer 20 with the underlying hardware. The middleware layer 30 includes an Intent Router Service (IR Service), as well as parallel standard data channels and atomic control channels. The atomic control channels bypass the audio processing link and directly connect to the underlying hardware abstraction layer 40. The Intent Router Service parses the intent of each audio control action in the semantic control action sequence, identifies the operation type of each audio control action, and distributes the action according to the operation type. This distribution based on the operation type includes: determining the category of the audio control action's operation type; if the category is a streaming media start / stop action, it is distributed to the standard data channel; if the category is an acoustic parameter adjustment action, it is distributed to the atomic control channel.

[0070] In some embodiments, the atomic control channel includes atomic service management (Atom Svc), which includes a mapping table maintenance service and various atomic services. The atomic service management receives acoustic parameter adjustment actions from the decision layer 20 and maps each acoustic parameter adjustment action to a specific hardware control parameter and the node identifier of the corresponding audio processing node by looking up the mapping table. The mapping table maintenance service maintains the mapping from the action intent to the node identifier (Node ID) of the audio processing engine and a set of adjustable parameters. For each acoustic parameter adjustment action, an atomic parameter instruction is generated based on the mapped hardware control parameters and node identifier. The generated atomic parameter instruction is directly communicated to the hardware abstraction layer 40 via socket communication.

[0071] Specifically, for each acoustic parameter adjustment action, the atomic service corresponding to the adjustment subclass of the acoustic parameter adjustment action in the atomic control channel is invoked. The acoustic parameter adjustment action is mapped to hardware control parameters and the node identifier of the audio processing node according to the lookup mapping table. Based on the hardware control parameters and the node identifier, the atomic parameter instruction corresponding to the acoustic parameter adjustment action is generated.

[0072] In some embodiments, the atomic service generates an atomic parameter instruction corresponding to the acoustic parameter adjustment action based on the hardware control parameters and the node identifier, including: querying the current audio processing topology of the vehicle audio system; for the acoustic parameter adjustment action in the control action sequence, generating a modification instruction for the target node through the atomic service corresponding to the adjustment subclass of the acoustic parameter adjustment action based on the current audio processing topology, the hardware control parameters corresponding to the acoustic parameter adjustment action, and the node identifier, and using the modification instruction as the atomic parameter instruction corresponding to the acoustic parameter adjustment action.

[0073] For example, the modification instructions generated by adjusting the acoustic parameters in the control action sequence include the following: Modification instruction 1: Configure the mixer node to reduce the music channel gain to -20dB; Modification instruction 2: Configure the routing node B to change the output of the call channel from the whole vehicle speakers to the passenger headrest; Modification instruction 3: Configure the post-processing node C to activate the human voice enhancement algorithm.

[0074] Step S204: The hardware abstraction layer 40 receives the audio data stream output from the standard data channel and writes the audio data stream into the hardware processor layer 50; the hardware abstraction layer 40 receives the atomization parameter instruction output from the atomization control channel, performs protocol conversion on the atomization parameter instruction output from the atomization control channel to obtain the remote procedure call request, and the hardware abstraction layer 40 initiates a remote procedure call request to the hardware processor layer.

[0075] In Android-based in-vehicle audio systems, the standard data channel transmits audio data streams to the hardware abstraction layer via HIDL (HAL Interface Definition Language) or AIDL (Android Interface Definition Language). Both HIDL and AIDL are interface description languages ​​used to define the interface between the hardware abstraction layer and the upper-level framework, aiming to achieve inter-process communication (IPC).

[0076] In some embodiments, the hardware abstraction layer 40 is responsible for protocol conversion. The hardware abstraction layer 40 includes a first interface and a second interface. The first interface is an extended interface, referring to a dedicated channel defined by the manufacturer to implement functions not covered by standard specifications, in addition to the essential interfaces stipulated by industry standards (such as USB, PCIe, and audio specifications). The hardware abstraction layer 40 receives and processes atomic parameter instructions transmitted by atomic services within the atomic control channel via socket communication through the first interface. It then encapsulates the atomic parameter instructions (Java instructions) into a remote procedure call request through the first interface and sends it to the hardware processor layer 50, enabling direct communication between the hardware abstraction layer 40 and the underlying hardware processor layer 50. The hardware abstraction layer 40 initiates a remote call request carrying atomic parameter instructions. This request penetrates the virtualization layer (Hypervisor) of the vehicle's infotainment system through a shared memory mechanism. Taking an Android system as an example, this bypasses the data processing queue of the audio mixing service (Audio Flinger) in the operating system's audio processing chain. The second interface is a standard interface, which refers to an interface publicly defined by an industry standards organization that does not depend on a specific vendor. The second interface receives the audio data stream output from the standard data channel and processes the reading and writing of the audio data stream from the standard data channel.

[0077] Step S205: In response to the remote procedure call request, the hardware processor layer 50 changes the audio processing topology of the vehicle audio system according to the audio data stream and the atomic parameter instructions carried in the remote procedure call request, so as to change the audio effect in the cabin.

[0078] In some embodiments, the hardware processor layer 50 runs an audio processing engine. In response to a remote procedure call request, the audio processing engine locates the corresponding audio processing node in its audio processing topology. It then modifies the value of the control buffer corresponding to the audio processing node, changing the processing logic of the audio digital signal processor in the hardware processor layer to reconstruct the audio processing topology. This audio processing topology reconstruction allows the mixing matrix to complete coefficient switching within milliseconds. The front passenger enjoys a clear headrest-based call experience.

[0079] AWE (Audio Weaver), as an industrial-grade standard engine for embedded real-time audio processing, uses an Audio Digital Signal Processor (ADSP) as its hardware platform. By controlling the buffer, it configures parameters and manages the status of each audio processing node in the audio processing topology, dynamically adjusting the data flow topology and algorithm combination relationship between audio processing nodes, thereby realizing the real-time online reconstruction of the in-vehicle cabin audio processing map.

[0080] Specifically, the AWE engine is mounted on a dedicated audio ADSP processor and constructs an audio processing topology consisting of multiple audio processing nodes. Each audio processing node is associated with a control buffer, and the corresponding audio processing node's operating parameters and working status are updated in real time by modifying the values ​​in the control buffer. The AWE engine dynamically adjusts the audio data flow and the topological connection relationship of the audio processing nodes based on the node status, realizing the dynamic reconstruction of the audio processing map, thereby directly changing the processing logic of the audio digital signal processor.

[0081] In some embodiments, when the processing logic of the audio digital signal processor in the hardware processor layer is changed, the background music fades out smoothly, while the navigation announcement is played from the driver's side headrest.

[0082] The following example illustrates an emergency conference call taking place in the passenger seat on a highway. In this situation, it's necessary to create a private audio space for the passenger without interrupting the driver's navigation and background music.

[0083] Input of multimodal perception information: The DMS module processes cockpit camera data, detects that the front passenger is using a mobile phone to make a call, and simultaneously triggers an audio focus request via the Bluetooth phone app, while also sensing the vehicle speed as 100 km / h. The multimodal perception information is dynamically constructed into structured prompt words, and the feature representation sequence generated from these prompt words is then fed into a large language model running on the NPU.

[0084] The large language model running on the NPU performs inference to determine the current user intent. It determines that the current user need is "co-pilot conference" mode. Based on the "co-pilot conference" mode, it outputs a sequence of control actions: open the co-pilot headrest, lower the music volume, activate the voice enhancement algorithm, and play the navigation voice into the driver's headrest.

[0085] The Atom Service Management Module (Atom Svc) of the Atom Control Channel receives the above control action sequence, queries the current audio processing topology of the in-vehicle audio system, and calculates a set of atomic node modification instructions based on the current audio processing topology. This set of node modification instructions includes: Modification Instruction 1: Configure the mixer node to reduce the music channel gain to -20dB; Modification Instruction 2: Configure the routing node B to change the output of the call channel from all vehicle speakers to the passenger-side headrest; Modification Instruction 3: Configure the post-processing node C to activate the voice enhancement algorithm.

[0086] The atomic service located in the atomic control channel transmits the above set of atomic node modification instructions to the hardware abstraction layer via Socket communication.

[0087] The hardware abstraction layer initiates a remote procedure call request, which carries the aforementioned set of atomic node modification instructions. Through the shared memory mechanism, it penetrates the operating system's Hypervisor virtualization layer and bypasses the data processing queue of the AudioFlinger service.

[0088] When the AWE audio processing engine, running on the hardware processor layer, receives a remote procedure call request from the hardware abstraction layer, it locates the corresponding node in the audio processing topology in real time and directly changes the DSP's processing logic by modifying the values ​​in the control buffer. For example, it can dynamically change the coefficients of the mixer matrix or insert / bypass an algorithm node (such as a vocal enhancement node) in real time in the audio path, thereby changing the audio effect in the passenger seat within milliseconds.

[0089] Based on the same inventive concept, embodiments of the present invention provide an in-vehicle audio control device. Figure 3 A schematic diagram of the structure of an in-vehicle audio control device according to some embodiments of the present invention is shown. For example... Figure 3 As shown, the in-vehicle audio control device includes: a data acquisition unit 301, used to acquire a feature representation sequence characterizing the vehicle state and the environment inside the vehicle cabin; an intent parsing unit 302, used to call a trained large language model to parse the feature representation sequence to obtain a control action sequence for the in-vehicle audio system, the control action sequence including multiple audio control actions; a data processing unit 303, used to distribute each audio control action in the control action sequence to a parallel standard data channel and an atomic control channel according to the operation type of each audio control action in the control action sequence, and process the audio control actions of the corresponding operation type through the standard data channel and the atomic control channel to output an audio data stream and atomic parameter instructions for the in-vehicle audio system; and a topology changing unit 304, used to change the audio processing topology of the in-vehicle audio system according to the audio data stream and the atomic parameter instructions to change the audio effect inside the cabin.

[0090] In some embodiments, the data acquisition unit 301 is configured to: acquire vehicle status data and cabin environment data characterizing the status of occupants in the cabin, wherein the vehicle status data includes vehicle body status parameters and status information of in-vehicle audio applications, and the cabin environment data includes at least one of voice commands, position, and posture of occupants in the cabin; construct structured prompt words based on the vehicle status data and the cabin environment data; and convert the structured prompt words into the feature representation sequence.

[0091] In some embodiments, the data acquisition unit 301 is configured to: acquire voice commands from occupants in the cabin via a microphone; acquire images of the cabin via a camera and identify the position and / or posture of each occupant based on the images; acquire at least one of the following vehicle status parameters via a CAN bus and / or Ethernet: vehicle speed, gear position, window status, and ambient noise level data; and acquire status information of various in-vehicle audio applications running on the vehicle terminal.

[0092] In some embodiments, the intent parsing unit 302 is configured to: invoke the large language model to perform semantic analysis and reasoning on the feature representation sequence to determine the current user intent of each person in the cabin or the target person; and invoke the large language model to process the current user intent to obtain the control action sequence.

[0093] In some embodiments, the operation types of the audio control actions include streaming media start / stop actions and acoustic parameter adjustment actions; the data processing unit 303 includes a distribution subunit, configured to: distribute the streaming media start / stop actions in the control action sequence to the standard data channel; and distribute the acoustic parameter adjustment actions in the control action sequence to the atomized control channel.

[0094] In some embodiments, the data processing unit 303 includes a dual-channel distribution unit, configured to: respond to the streaming media start / stop action in the control action sequence by using a standard audio processing link within the standard data channel to control the transmission of audio data streams; and process each acoustic parameter adjustment action in the control action sequence by using a corresponding atomic service within the atomic control channel to obtain the atomic parameter instruction corresponding to the audio control action.

[0095] In some embodiments, the acoustic parameter adjustment class includes multiple adjustment subclasses, and the atomic control channel includes multiple atomic services corresponding to the multiple adjustment subclasses; the dual-channel subunit is used to: call the atomic service corresponding to the adjustment subclass of the acoustic parameter adjustment action in the atomic control channel, map the acoustic parameter adjustment action to hardware control parameters and the node identifier of the audio processing node, and generate the atomic parameter instruction corresponding to the acoustic parameter adjustment action according to the hardware control parameters and the node identifier.

[0096] In some embodiments, the dual-channel subunit is configured to: query the current audio processing topology of the vehicle audio system; and, for the acoustic parameter adjustment action in the control action sequence, generate a modification instruction for the target node based on the current audio processing topology, the hardware control parameters corresponding to the acoustic parameter adjustment action, and the node identifier through the atomic service corresponding to the adjustment subclass of the acoustic parameter adjustment action, and use the modification instruction as the atomic parameter instruction corresponding to the acoustic parameter adjustment action.

[0097] In some embodiments, the topology change unit 304 is configured to: encapsulate the atomic parameter instructions output by the atomic control channel into a remote procedure call request; modify the state of the target node of the vehicle audio system according to the remote procedure call request, so as to reconstruct the audio processing topology in the cabin, so as to improve the audio effect in the vehicle cabin.

[0098] In some embodiments, the audio control device is applied to an in-vehicle infotainment system. The audio processing architecture of the in-vehicle infotainment system includes a multimodal perception layer, a decision layer, a middleware layer, a hardware abstraction layer, and a hardware processor layer arranged from top to bottom. The standard data channel and the atomic control channel are located in the middleware layer.

[0099] In some embodiments, the data acquisition unit is located in the multimodal perception layer, and the data acquisition unit 301 located in the multimodal perception layer acquires the feature representation sequence representing the vehicle state and the state of the people in the cabin; The intent parsing unit 302 is located in the decision layer. The intent parsing unit 302 in the decision layer calls the trained large language model to parse the feature representation sequence to obtain the control action sequence for the vehicle audio system. The control action sequence includes multiple audio control actions. The data processing unit 303 is located in the middleware layer. According to the operation type of each audio control action in the control action sequence, the data processing unit located in the middleware layer distributes each audio control action in the control action sequence to a parallel standard data channel and an atomic control channel. The audio control actions of the corresponding operation type are processed through the standard data channel and the atomic control channel to output an audio data stream and atomic parameter instructions for the vehicle audio system. The topology changing unit 304 is used to change the audio processing topology of the vehicle audio system based on the communication between the hardware abstraction layer and the hardware processor layer, according to the audio data stream and the atomic parameter instructions, so as to change the audio effect in the cabin.

[0100] In some embodiments, the hardware abstraction layer includes a first interface and a second interface. The topology changing unit 304 is configured to: receive atomic parameter instructions transmitted by atomic services within the atomic control channel via socket communication through the first interface of the hardware abstraction layer, and encapsulate the atomic parameter instructions into a remote procedure call request; receive audio data streams output from the standard data channel through the second interface of the hardware abstraction layer; respond to the remote procedure call request through an audio processing engine running on the hardware processor layer, locate the corresponding node in the audio processing topology of the audio processing engine; and modify the processing logic of the audio digital signal processor in the hardware processor layer by modifying the value of the control buffer corresponding to the node, thereby reconstructing the audio processing topology.

[0101] Based on the same inventive concept, embodiments of the present invention provide an electronic device. Figure 4 A schematic diagram of an electronic device according to some embodiments of the present invention is shown. (Reference) Figure 4 As shown, the electronic device includes one or more memories 404, one or more processors 402, and at least one piece of program code stored in the one or more memories 404. The at least one piece of program code is loaded and executed by the one or more processors 402 to implement the vehicle audio control method according to any embodiment of the present invention.

[0102] Among them, Figure 4 In this document, a bus architecture (represented by bus 400) is used. Bus 400 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 402 and memory represented by memory 404. Bus 400 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 405 provides an interface between bus 400 and receiver 401 and transmitter 403. Receiver 401 and transmitter 403 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 402 is responsible for managing bus 400 and general processing, while memory 404 can be used to store data used by processor 402 during operation.

[0103] Based on the same inventive concept, embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the vehicle audio control method described in any embodiment of the present invention.

[0104] According to one or more embodiments of the present invention, at least the following technical effects or advantages are achieved: 1. Traditional solutions rely on static XML configuration files, which can only switch between a few preset modes (such as music and navigation). This invention combines the semantic understanding capabilities of a large language model with the dynamic reconstruction of the audio processing topology by the AWE audio processing engine, achieving real-time mapping of topology parameters from user intent to the underlying hardware processing layer. This enables the generation of precise audio control strategies in real time based on multimodal information such as vehicle status, occupant position and posture, and the status of in-vehicle audio applications. This breaks through the limitations of traditional static configuration file-based strategies and significantly improves the intelligence level of cockpit audio.

[0105] 2. By constructing an atomic control path parallel to the middleware layer and utilizing RPC technology and shared memory mechanisms, the cumbersome audio processing chain of the operating system was successfully bypassed. This enables the AI ​​agent to control the underlying hardware resources with deterministic low latency characteristics, ensuring millisecond-level instantaneous response in critical scenarios such as noise reduction and sound zone isolation, and solving the technical problems of lengthy control chains and slow response in traditional architectures.

[0106] 3. Completely decouple the underlying signal processing capabilities from the upper-layer business logic. At the same time, standardize audio capabilities and encapsulate discrete DSP algorithm modules into standardized atomic interfaces (such as volume atomic services and sound effect atomic services). This allows AI agents to reuse underlying computing power in a standardized manner, avoiding the waste of resources from repeatedly developing underlying drivers for different in-vehicle audio applications, and significantly improving the maintainability and scalability of the system.

[0107] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable code.

[0108] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer instructions. These computer instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0109] These computer instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer instructions may also be loaded onto a computer or other programmable execution data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0111] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0112] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A vehicle-mounted audio control method, characterized in that, include: Obtain feature representation sequences that characterize the vehicle state and the state of the occupants in the cabin; The trained large language model is invoked to parse the feature representation sequence to obtain a control action sequence for the vehicle audio system, the control action sequence including multiple audio control actions; Based on the operation type of each audio control action in the control action sequence, each audio control action in the control action sequence is distributed to a parallel standard data channel and an atomic control channel. The audio control actions of the corresponding operation type are processed through the standard data channel and the atomic control channel to output an audio data stream and atomic parameter instructions for the vehicle audio system. The audio processing topology of the in-vehicle audio system is altered according to the audio data stream and the atomication parameter instructions to change the audio effect in the cabin.

2. The vehicle audio control method as described in claim 1, characterized in that, The acquisition of the feature representation sequence characterizing the vehicle state and the state of the occupants in the cabin includes: Acquire vehicle status data and cabin environment data characterizing the status of occupants in the cabin. The vehicle status data includes vehicle body status parameters and status information of in-vehicle audio applications. The cabin environment data includes at least one of the voice commands, position, and posture of occupants in the cabin. Structured prompts are constructed based on the vehicle status data and the cabin environment data; The structured cue words are converted into the feature representation sequence.

3. The vehicle audio control method as described in claim 2, characterized in that, The acquisition of vehicle status data and cabin environment data characterizing the status of occupants includes: Voice commands from occupants in the cabin are collected via microphone; The system captures images inside the cockpit using a camera and identifies the position and / or posture of each person inside the cockpit based on these images. At least one of the following vehicle status parameters is obtained via CAN bus and / or Ethernet: vehicle speed, gear position, window status, and ambient noise level data. Obtain the status information of each in-vehicle audio application running on the vehicle's infotainment system.

4. The vehicle audio control method as described in claim 1, characterized in that, The trained large language model is invoked to parse the feature representation sequence to obtain a control action sequence for the in-vehicle audio system, including: The large language model is invoked to perform semantic analysis and reasoning on the feature representation sequence to determine the current user intent of each person in the cabin or the target person; The large language model is invoked to process the current user intent in order to obtain the control action sequence.

5. The vehicle audio control method as described in claim 1, characterized in that, The audio control actions include streaming media start / stop actions and acoustic parameter adjustment actions. The step of distributing each audio control action in the control action sequence to parallel standard data channels and atomic control channels according to the operation type of each audio control action in the control action sequence includes: Distribute the streaming media start / stop actions in the control action sequence to the standard data channel; The acoustic parameter adjustment actions in the control action sequence are distributed to the atomized control channel.

6. The method as described in claim 5, characterized in that, The process of processing audio control actions of corresponding operation types through the standard data channel and the atomic control channel to output audio data streams and atomic parameter commands for the vehicle audio system includes: For the streaming media start / stop action in the control action sequence, the standard audio processing link in the standard data channel responds to the streaming media start / stop action to control the transmission of audio data stream; For each acoustic parameter adjustment action in the control action sequence, the acoustic parameter adjustment action is processed by the corresponding atomic service in the atomic control channel to obtain the atomic parameter instruction corresponding to the audio control action.

7. The vehicle audio control method as described in claim 5, characterized in that, The acoustic parameter adjustment class includes multiple adjustment subclasses, and the atomization control channel includes multiple atomic services corresponding to the multiple adjustment subclasses; The process of processing the acoustic parameter adjustment action through the corresponding atomic service within the atomic control channel to obtain the atomic parameter instruction corresponding to the audio control action includes: The atomic service corresponding to the adjustment subclass of the acoustic parameter adjustment action within the atomic control channel is invoked to map the acoustic parameter adjustment action to hardware control parameters and the node identifier of the audio processing node, and the atomic parameter instruction corresponding to the acoustic parameter adjustment action is generated according to the hardware control parameters and the node identifier.

8. The vehicle audio control method as described in claim 7, characterized in that, The atomic service generates atomic parameter instructions corresponding to the acoustic parameter adjustment action based on the hardware control parameters and the node identifier, including: Query the current audio processing topology of the vehicle audio system; For the acoustic parameter adjustment action in the control action sequence, the atomic service corresponding to the adjustment subclass of the acoustic parameter adjustment action generates a modification instruction for the target node based on the current audio processing topology, the hardware control parameters corresponding to the acoustic parameter adjustment action, and the node identifier. The modification instruction is then used as the atomic parameter instruction corresponding to the acoustic parameter adjustment action.

9. The vehicle audio control method as described in claim 1, characterized in that, The step of changing the audio processing topology of the in-vehicle audio system according to the audio data stream and the atomication parameter instructions to change the audio effect in the cabin includes: The atomization parameter instructions output by the atomization control channel are encapsulated as a remote procedure call request; The state of the target node of the in-vehicle audio system is modified according to the remote procedure call request to reconstruct the audio processing topology in the cabin, so as to improve the audio effect in the vehicle cabin.

10. The vehicle audio control method as described in any one of claims 1-9, characterized in that, The audio control method is applied to the vehicle's infotainment system. The audio processing architecture of the infotainment system includes a multimodal perception layer, a decision layer, a middleware layer, a hardware abstraction layer, and a hardware processor layer arranged from top to bottom. The standard data channel and the atomic control channel are located in the middleware layer.

11. The vehicle audio control method as described in claim 10, characterized in that, The multimodal perception layer acquires feature representation sequences characterizing the vehicle state and the state of the occupants in the cabin; The decision layer calls the trained large language model to parse the feature representation sequence to obtain a control action sequence for the vehicle audio system, the control action sequence including multiple audio control actions; The middleware layer distributes each audio control action in the control action sequence to a parallel standard data channel and an atomic control channel according to the operation type of each audio control action in the control action sequence. The standard data channel and the atomic control channel process the audio control actions of the corresponding operation type to output an audio data stream and atomic parameter instructions for the vehicle audio system. Based on the communication between the hardware abstraction layer and the hardware processor layer, the audio processing topology of the in-vehicle audio system is changed according to the audio data stream and the atomic parameter instructions to change the audio effect in the cabin.

12. The vehicle audio control method as described in claim 11, characterized in that, The hardware abstraction layer includes a first interface and a second interface. The communication between the hardware abstraction layer and the hardware processor layer, based on the audio data stream and the atomic parameter instructions, modifies the audio processing topology of the in-vehicle audio system to change the audio effect in the cabin, including: The first interface of the hardware abstraction layer receives atomic parameter instructions transmitted by the atomic service in the atomic control channel via socket communication, and encapsulates the atomic parameter instructions into a remote procedure call request; The audio data stream output from the standard data channel is received by the second interface of the hardware abstraction layer; In response to the remote procedure call request, the audio processing engine running on the hardware processor layer locates the corresponding node in the audio processing topology of the audio processing engine; By modifying the value of the control buffer corresponding to the node, the processing logic of the audio digital signal processor in the hardware processor layer is changed to reconstruct the audio processing topology.

13. An audio control device for a vehicle cockpit, characterized in that, include: The data acquisition unit is used to acquire feature representation sequences that characterize the vehicle state and the environment inside the vehicle cabin. The intent parsing unit is used to call the trained large language model to parse the feature representation sequence to obtain the control action sequence for the vehicle audio system, the control action sequence including multiple audio control actions; The data processing unit is configured to distribute each audio control action in the control action sequence to a parallel standard data channel and an atomic control channel according to the operation type of each audio control action in the control action sequence, and process the audio control actions of the corresponding operation type through the standard data channel and the atomic control channel to output an audio data stream and atomic parameter instructions for the vehicle audio system. The topology changing unit is used to change the audio processing topology of the vehicle audio system according to the audio data stream and the atomization parameter instructions, so as to change the audio effect in the cabin.

14. An electronic device, characterized in that, The device includes one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, and the at least one piece of program code is loaded and executed by the one or more processors to implement the in-vehicle audio control method according to any one of claims 1-12.

15. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the in-vehicle audio control method as described in any one of claims 1-12.

Images