Prompt tone generation method and device, vehicle and storage medium

Abstract

The invention relates to the technical field of warning tones of vehicles, in particular to a warning tone generation method and device, a vehicle and a storage medium, and the method comprises the steps: obtaining monaural warning tone data of the vehicle and space state data of a target object around the vehicle; according to the spatial state data of the target object, encoding the monaural prompt tone data into three-dimensional space audio data, and acquiring loudspeaker data of the vehicle; and generating audio middleware data based on the loudspeaker data and the three-dimensional space audio data, and rendering the audio middleware data to generate a prompt tone of the vehicle. Therefore, the problems that the prompt tone cannot meet diversified prompt tone scene requirements, the expansibility is poor and the like in the prior art are solved.

Description

Technical Field

[0001] This application relates to the field of vehicle alert tone technology, and in particular to a method, device, vehicle, and storage medium for generating alert tones. Background Technology

[0002] With the development of vehicle intelligence, intelligent prompts with directional features can effectively guide the driver's attention, reduce the driver's reliance on the instrument panel or central control, and significantly improve driving safety and comfort. For example, when there is an obstacle on the left rear, the prompt sound comes from the left rear; when overtaking on the left side, the prompt sound smoothly transitions from the left rear to the left front.

[0003] The implementation of such prompts in related technologies is mainly based on surround sound technology, which pre-prepares audio files for different scenarios and directions. When a sound is emitted, the corresponding file is selected for playback. However, the number of pre-prepared audio files is limited, which cannot meet the needs of diverse prompt scenarios. Furthermore, when the speaker channel layout is different, the pre-prepared audio files need to be re-rendered, resulting in weak system scalability. Summary of the Invention

[0004] This application provides a method, apparatus, vehicle, and storage medium for generating prompt sounds, in order to solve the problems in related technologies such as prompt sounds failing to meet the needs of diverse prompt sound scenarios and having poor scalability.

[0005] The first aspect of this application provides a method for generating a prompt sound, comprising the following steps: acquiring mono prompt sound data of a vehicle and spatial state data of objects surrounding the vehicle; encoding the mono prompt sound data into three-dimensional spatial audio data based on the spatial state data of the objects, and acquiring speaker data of the vehicle; generating audio middleware data based on the speaker data and the three-dimensional spatial audio data, and rendering the audio middleware data to generate the vehicle's prompt sound.

[0006] Optionally, in one embodiment of this application, the spatial state data includes at least one of orientation data, motion duration, and motion pattern. Encoding the mono prompt data into three-dimensional spatial audio data based on the spatial state data of the target object includes: identifying the type of the target object; if the target object is of a first type, dividing the motion duration into multiple sampling points, determining a target interpolation method based on the motion pattern, calculating the orientation data for each sampling point based on the target interpolation method, and encoding the mono prompt data into three-dimensional spatial audio data based on the orientation data; if the target object is of a second type, encoding the mono prompt data into three-dimensional spatial audio data based on the orientation data.

[0007] Optionally, in one embodiment of this application, before acquiring the mono tone data of the vehicle and the spatial state data of the target objects around the vehicle, the method further includes: acquiring the target coordinate system of the vehicle, wherein the target coordinate system is constructed with the driver's seat of the vehicle as the origin, the front of the driver's seat as the positive X-axis, the right side of the driver's seat as the positive Y-axis, and the plane perpendicular to the X-axis and Y-axis as the positive Z-axis; and determining the spatial state data of the target objects based on the target coordinate system.

[0008] Optionally, in one embodiment of this application, the orientation data includes at least one of azimuth angle, elevation angle, and distance. Encoding the mono prompt data into three-dimensional spatial audio data based on the orientation data includes: calling the corresponding head-related transfer function filter according to the azimuth angle and / or elevation angle; filtering the mono prompt data using the head-related transfer function filter; attenuating the filtered mono prompt data based on the distance and target attenuation curve; and encoding the attenuated mono prompt data into three-dimensional spatial audio data.

[0009] Optionally, in one embodiment of this application, after rendering the audio middleware data, the method further includes: identifying the rendering result; if the rendering result is a rendering failure, then outputting a mono prompt corresponding to the mono prompt data.

[0010] Optionally, in one embodiment of this application, before acquiring the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle, the method further includes: identifying a vehicle prompt sound generation identifier; if the prompt sound generation identifier is a first identifier, then acquiring pre-rendered audio middleware data stored in the vehicle, rendering the pre-rendered audio middleware data to generate the vehicle's prompt sound; otherwise, acquiring the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle.

[0011] Optionally, in one embodiment of this application, obtaining pre-rendered audio middleware data stored in the vehicle includes: identifying multiple mono prompt tone data in a preset safety prompt tone list of the vehicle when the vehicle is started; obtaining multiple directional data covered by each mono prompt tone data; and pre-rendering each mono prompt tone data based on speaker data and multiple directional data corresponding to each mono prompt tone to obtain multiple pre-rendered audio middleware data.

[0012] A second aspect of this application provides a prompt sound generation device, comprising: an acquisition module for acquiring mono prompt sound data of a vehicle and spatial state data of targets around the vehicle; an encoding module for encoding the mono prompt sound data into three-dimensional spatial audio data based on the spatial state data of the targets, and acquiring speaker data of the vehicle; and a generation module for generating audio middleware data based on the speaker data and the three-dimensional spatial audio data, and rendering the audio middleware data to generate a vehicle prompt sound.

[0013] Optionally, in one embodiment of this application, the spatial state data includes at least one of orientation data, motion duration, and motion pattern. The encoding module is further configured to: identify the type of the target object; if the type of the target object is a first type, divide the motion duration into multiple sampling points, determine the target interpolation method based on the motion pattern, calculate the orientation data of each sampling point based on the target interpolation method, and encode the mono prompt data into three-dimensional spatial audio data based on the orientation data; if the type of the target object is a second type, encode the mono prompt data into three-dimensional spatial audio data based on the orientation data.

[0014] Optionally, in one embodiment of this application, it further includes: a construction module, used to acquire the vehicle's target coordinate system before acquiring the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle, wherein the target coordinate system is constructed with the driver's seat of the vehicle as the origin, the front of the driver's seat as the positive X-axis, the right side of the driver's seat as the positive Y-axis, and the plane perpendicular to the X-axis and Y-axis as the positive Z-axis; and to determine the spatial state data of the target objects based on the target coordinate system.

[0015] Optionally, in one embodiment of this application, the orientation data includes at least one of azimuth angle, elevation angle, and distance. The encoding module is further configured to: call the corresponding head-related transfer function filter according to the azimuth angle and / or elevation angle, the head-related transfer function filter performs filtering processing on the mono prompt data; perform attenuation processing on the filtered mono prompt data based on the distance and target attenuation curve; and encode the attenuated mono prompt data into three-dimensional spatial audio data.

[0016] Optionally, in one embodiment of this application, it further includes: an output module, used to identify the rendering result after rendering the audio middleware data; if the rendering result is a rendering failure, then outputting a mono prompt tone corresponding to the mono prompt tone data.

[0017] Optionally, in one embodiment of this application, it further includes: a rendering module, configured to identify a vehicle prompt sound generation identifier before acquiring the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle; if the prompt sound generation identifier is a first identifier, then acquire the pre-rendered audio middleware data stored in the vehicle, render the pre-rendered audio middleware data to generate the vehicle's prompt sound, otherwise acquire the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle.

[0018] Optionally, in one embodiment of this application, the rendering module is further configured to: identify multiple mono prompt tone data in the vehicle's preset safety prompt tone list when the vehicle is started; obtain multiple directional data covered by each mono prompt tone data; and pre-render each mono prompt tone data based on the speaker data and the multiple directional data corresponding to each mono prompt tone to obtain multiple pre-rendered audio middleware data.

[0019] A third aspect of this application provides a vehicle, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to perform the prompt sound generation method as described in the above embodiments.

[0020] A fourth aspect of this application provides a computer-readable storage medium having a computer program or instructions stored thereon, which is executed by a processor to perform the prompt sound generation method as described in the above embodiments.

[0021] A fifth aspect of this application provides a computer program product, including a computer program or instructions, which, when executed, implement the prompt sound generation method as described in the above embodiments.

[0022] Therefore, this application has at least the following beneficial effects: This application embodiment encodes mono alert sound data into three-dimensional spatial audio data based on the spatial state data of targets around the vehicle, giving the audio a sense of direction and distance. Furthermore, it generates audio middleware data based on the vehicle's speaker data and the three-dimensional spatial audio data, and renders the audio middleware data to generate the vehicle's alert sound. This eliminates the need for pre-set multi-scene, multi-directional audio files. By processing mono data and spatial state data in real time, it flexibly generates alert sounds for various scenarios and can adjust audio allocation according to the speaker data of different vehicle models, improving scalability. Therefore, it solves the technical problems in related technologies, such as the inability of alert sounds to meet the diverse needs of alert sound scenarios and poor scalability.

[0023] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0024] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a flowchart of a prompt sound generation method provided according to an embodiment of this application; Figure 2 This is a flowchart of a pre-rendered prompt sound provided according to an embodiment of this application; Figure 3 This is an architecture diagram of the panoramic sound prompt generation engine provided according to an embodiment of this application; Figure 4 This is a flowchart illustrating the real-time rendering of prompt sounds according to an embodiment of this application; Figure 5 This is a schematic diagram of the dynamic trajectory provided according to the embodiments of this application; Figure 6 This is an example diagram of a prompt sound generation device provided according to an embodiment of this application; Figure 7 This is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0026] Before describing the solution of this application, let me first introduce the relevant technologies of this application.

[0027] With the development of vehicle intelligence, more intelligent alert sounds can bring users a better experience. For example, when the reversing alarm sounds, if there is an obstacle on the left rear, the user will perceive the alarm sound from the left rear; while driving, if a vehicle approaches quickly from the left, the alert sound will gradually transition from the left rear to the left front; and during navigation, left turn and right turn prompts will be provided. Directional alert sounds can effectively guide the driver's attention, allowing the driver to more naturally and smoothly notice the areas that need attention, without having to look at the instrument panel or center console after hearing an alarm or alert sound, greatly improving driving safety and comfort.

[0028] Currently, the implementation of related technologies for this scenario is based on surround sound technology, which pre-prepares many audio files. These audio files correspond to different scenarios and directions of prompts. When a sound request is received, the corresponding audio file is selected for playback. However, based on a limited number of pre-prepared audio files, the content cannot meet the diverse prompt requirements. In addition, if the speaker channel layout is different, the pre-prepared sounds need to be re-rendered and generated, resulting in weak system scalability. Some related technologies also use only one original file for prompts with the same timbre. After receiving a sound request, the gain of different channels is adjusted according to different locations to achieve a similar effect. However, the sound immersion is poor, and the simulation effect is poor for moving or wide sound sources.

[0029] In summary, the relevant technologies have the following problems: they rely on pre-made audio files, which cannot meet the needs of diverse scenarios, and require re-rendering when adapting to different speaker layouts, resulting in poor scalability; the simulation method based on surround sound technology lacks immersion, the directional transition effect of dynamic sound sources (such as moving obstacle prompts) is abrupt, and the simulation accuracy of width is low; the real-time generation of safety prompt sounds has the risk of delay, and it is impossible to balance rendering efficiency and reliability.

[0030] To address this, this application provides a method for generating prompt sounds, which can generate prompt sounds with accurate orientation and dynamic trajectory in real time, adapting to the speaker layout of different vehicle models, improving driving safety and interactive experience, and solving at least one of the above technical problems.

[0031] Specifically, Figure 1 This is a flowchart illustrating a prompt sound generation method provided in an embodiment of this application.

[0032] like Figure 1 As shown, the method for generating the prompt sound includes the following steps: In step S101, the mono audio data of the vehicle and the spatial state data of the target objects around the vehicle are acquired.

[0033] Among them, mono prompt sound data can also be called raw audio, which is a raw prompt sound file containing only a single audio channel without spatial processing, such as reversing alarm sound or collision warning sound, and the format can be WAV format; the target object can be obstacles around the vehicle, overtaking vehicles, etc.; spatial state data includes at least one of orientation data, motion duration and motion mode, and the motion mode can be uniform acceleration, uniform speed, uniform deceleration, etc. In one embodiment of this application, before acquiring the mono tone data of the vehicle and the spatial state data of the target objects around the vehicle, the method further includes: acquiring the target coordinate system of the vehicle, wherein the target coordinate system is constructed with the driver's seat of the vehicle as the origin, the front of the driver's seat as the positive X-axis, the right side of the driver's seat as the positive Y-axis, and the plane perpendicular to the X-axis and Y-axis as the positive Z-axis; and determining the spatial state data of the target objects based on the target coordinate system.

[0034] The target coordinate system is used to accurately describe the spatial position of the target object. It can be constructed with the driver's seat as the origin, the positive X-axis in front of the driver's seat, the positive Y-axis to the right of the driver's seat, and the positive Z-axis perpendicular to the XY plane. Alternatively, a specific coordinate system can be constructed according to the specific situation. This embodiment is based on this coordinate system.

[0035] It is understood that the embodiments of this application can obtain the target coordinate system of the vehicle and determine the spatial state data of the target based on the target coordinate system. By constructing a unified target coordinate system, the spatial state data of different scenarios and different targets can have a unified measurement benchmark, avoiding confusion in orientation calculation caused by inconsistent coordinate systems, ensuring the accuracy of three-dimensional audio encoding. At the same time, the target coordinate system can make the subsequent spatial state data more accurate, so that the orientation error of the subsequently generated three-dimensional prompt sound is controlled within ±5°, allowing the driver to perceive the position of the target more accurately.

[0036] Furthermore, it should be noted that the spatial state data of the target object in the embodiments of this application can be obtained in real time based on the target coordinate system, or it can be directly input based on other application layer software, but the spatial state data of the target object is based on the target coordinate system.

[0037] In one embodiment of this application, before acquiring the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle, the method further includes: identifying a vehicle prompt sound generation identifier; if the prompt sound generation identifier is a first identifier, then acquiring pre-rendered audio middleware data stored in the vehicle, rendering the pre-rendered audio middleware data to generate the vehicle's prompt sound; otherwise, acquiring the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle.

[0038] Among them, the prompt tone generation identifier is used to distinguish the prompt tone generation method, which can also be called the priority identifier. The first identifier corresponds to the security level, indicating that the real-time rendering requirement is low and the response speed requirement is high, and it is used to indicate security. The second identifier corresponds to the non-security level, indicating that the real-time rendering requirement is high. The pre-rendered audio middleware file can be stored in the vehicle's high-speed cache.

[0039] It is understood that the embodiments of this application can identify the vehicle's prompt sound generation identifier. If the prompt sound generation identifier is the first identifier, it indicates that the requirements for real-time rendering are low, but the requirements for response speed are high, and it is used to prompt safety. In this case, the pre-rendered audio middleware file stored in the vehicle is directly rendered to generate the vehicle's prompt sound, ensuring that there is no delay in playback when triggered, thereby improving response speed and reliability. If the prompt sound generation identifier is the second identifier, the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle are obtained, and the prompt sound is generated based on the mono prompt sound data and the spatial state data.

[0040] For example, when the vehicle starts, the reversing alarm is pre-rendered and stored. When the driver puts the vehicle in reverse, the pre-stored reversing alarm audio middleware file is directly called, rendered, and output through the speaker.

[0041] In one embodiment of this application, obtaining pre-rendered audio middleware data stored in the vehicle includes: identifying multiple mono prompt tone data in the vehicle's preset safety prompt tone list when the vehicle starts; obtaining multiple directional data covered by each mono prompt tone data; and pre-rendering each mono prompt tone data based on speaker data and the multiple directional data corresponding to each mono prompt tone to obtain multiple pre-rendered audio middleware data.

[0042] The preset safety warning tone list stores multiple predefined mono warning tone data, which can be multiple safety warning tones, such as reversing alarm tone, collision warning tone, lane departure warning tone, etc.

[0043] It is understood that, when the vehicle is started, the embodiments of this application can identify multiple mono prompt tone data in the vehicle's preset safety prompt tone list, and obtain multiple directional data covered by each mono prompt tone data. Based on the speaker data and the multiple directional data corresponding to each mono prompt tone, each mono prompt tone data is pre-rendered to obtain multiple pre-rendered audio middleware data, so as to ensure that the prompt tone can be output in a timely and accurate manner in all directions, and to ensure that the prompt tone responds quickly.

[0044] Specifically, the pre-rendering mechanism of this application embodiment is as follows: Figure 2 As shown, the following operations are performed when the cockpit system is first powered on: Configuration read: Reads the vehicle's speaker layout (e.g., 6 speakers) and safety warning tone list (i.e., the list of preset safety warning tones, such as reversing alarm, collision warning, etc. 5); Batch rendering: For each type of safety warning tone, pre-render audio data based on all possible directional parameters (such as reversing alarm covering -90°~90° directional range) and store it in DRAM (Dynamic Random Access Memory) cache; Verification and Update: Each time the cockpit system starts, the integrity of the pre-rendered data is verified. If the speaker configuration is changed (such as replacing the speakers after maintenance), the pre-rendering is automatically re-rendered. Triggered playback: For example, when ADAS (Advanced Driver Assistance Systems) triggers a safety warning, pre-rendered data is read directly from the cache and output through the audio middleware, with a response latency of ≤8ms.

[0045] In addition, it should be noted that, in addition to pre-rendering safety prompts, this application embodiment can also pre-render frequently used regular prompts (such as navigation turn signals). By partitioning storage (prioritizing commonly used directions), the cache usage and response speed are balanced, which is suitable for vehicle models with fixed speaker layouts.

[0046] In step S102, based on the spatial state data of the target object, the mono prompt sound data is encoded into three-dimensional spatial audio data, and the vehicle's speaker data is obtained.

[0047] Among them, the three-dimensional spatial audio data is an audio signal that has undergone spatial encoding processing and contains spatial information such as orientation and distance; the speaker data is used to describe relevant information about the vehicle's speakers, including the number of speakers, their location distribution, channel mapping, etc.

[0048] It is understood that, according to the embodiments of this application, the mono prompt tone data can be encoded into three-dimensional spatial audio data based on the spatial state data of the target object, giving the audio a sense of direction, and the vehicle's speaker data can be obtained so as to adapt to the vehicle's speaker distribution in the future.

[0049] In one embodiment of this application, encoding mono prompt data into three-dimensional spatial audio data based on the spatial state data of the target object includes: identifying the type of the target object; if the type of the target object is a first type, dividing the motion duration into multiple sampling points, determining a target interpolation method based on the motion pattern, calculating the orientation data of each sampling point based on the target interpolation method, and encoding the mono prompt data into three-dimensional spatial audio data based on the orientation data; if the type of the target object is a second type, encoding the mono prompt data into three-dimensional spatial audio data based on the orientation data.

[0050] The azimuth data includes at least one of azimuth, elevation, and distance, and the azimuth data can be in the form of (azimuth, elevation, distance); the first type of target is a target with motion attributes, also known as a dynamic target; the second type of target is a fixed target, also known as a static target.

[0051] It is understood that the embodiments of this application can identify the type of the target object in order to adopt different processing methods. Specifically, if the target object is of the first type, the motion duration is divided into multiple sampling points, and the orientation data of each sampling point is calculated by combining the motion mode selection interpolation method. Then, based on the orientation data of each sampling point, the mono prompt tone data is encoded. By dividing the sampling points and interpolating, the sound transitions smoothly with the target motion, thereby generating three-dimensional spatial audio data that changes with the movement of the target object, allowing the user to clearly perceive the direction and speed of the target object's motion. If the target object is of the second type, the mono prompt tone data is directly encoded into three-dimensional spatial audio data based on the orientation data.

[0052] The target interpolation method in this application embodiment can be determined based on the motion mode to ensure smooth audio transition corresponding to the motion trajectory of the target object. For example, if the motion mode is uniform acceleration, uniform speed, or uniform deceleration, it corresponds to the linear interpolation method. Other motion modes correspond to the Bézier curve interpolation method. Alternatively, a fixed linear interpolation method or a fixed Bézier curve interpolation method can be used directly, but the computational load is larger. The appropriate method can be selected based on the computing power of the vehicle's cabin.

[0053] For example, when a vehicle is driving, it detects a vehicle on the left overtaking it in a uniform acceleration mode (i.e., a first-type target). The spatial state data is a motion duration of 2 seconds, a motion mode of uniform acceleration, initial orientation data (-70°, 0°, 8m), and final orientation data (30°, 0°, 3m). The 2 seconds are divided into 100 sampling points (one every 20ms). Linear interpolation is used to calculate the azimuth angle (gradually transitioning from -70° to 30°) and distance (gradually decreasing from 8m to 3m) of each sampling point. Based on the orientation data of each sampling point, a mono prompt tone is encoded to generate dynamically changing three-dimensional spatial audio data. In the end, the user hears the prompt tone gradually and smoothly transitioning from the left rear to the left front, and can perceive that the overtaking vehicle is accelerating and approaching. If a fixed roadblock (i.e., a second type of target) is detected on the roadside ahead, with a azimuth data of (0°, 0°, 10m), the mono prompt tone is directly encoded based on this fixed azimuth data to generate a prompt tone from directly in front.

[0054] In one embodiment of this application, encoding mono prompt data into three-dimensional spatial audio data based on azimuth data includes: calling a corresponding head-related transfer function filter based on the azimuth angle and / or elevation angle, the head-related transfer function filter filtering the mono prompt data; attenuating the filtered mono prompt data based on distance and target attenuation curves; and encoding the attenuated mono prompt data into three-dimensional spatial audio data.

[0055] The HRTF (Head-Related Transfer Function) filter is used to simulate the human ear's perception of sound from different directions. Different azimuth and elevation angles correspond to different HRTF filter parameters. The target attenuation curve describes the law of sound attenuation as distance increases. The farther the distance, the greater the sound volume attenuation, and the attenuation of high-frequency components is more obvious. The target attenuation curve can be represented as attenuation coefficient = 1 / (distance). 2 ).

[0056] It is understood that, in this embodiment of the application, the corresponding head-related transfer function filter can be invoked according to the azimuth angle and / or elevation angle. The head-related transfer function filter filters the mono prompt data, so that the generated three-dimensional spatial audio data can accurately restore the orientation of the target object. Based on the distance and target attenuation curve, the filtered mono prompt data is attenuated, allowing the user to perceive the distance between the target object and the vehicle through the change of the prompt sound. The attenuated mono prompt data is encoded into three-dimensional spatial audio data. Through the three-dimensional encoding of azimuth filtering and distance attenuation, the prompt sound is closer to the sound propagation effect in the real environment, significantly enhancing the immersiveness of the sound and improving the driving interaction experience.

[0057] For example, when a vehicle detects a fixed obstacle 10 meters ahead, and the obstacle's spatial status data is (azimuth 0°, elevation 0°, distance 10m), the HRTF filter corresponding to azimuth 0° and elevation 0° is first invoked to filter the mono collision warning tone, giving the tone a forward directional characteristic. Then, based on the distance of 10m, the attenuation coefficient is adjusted according to 1 / (10... 2 The target attenuation curve is set to 0.01. The filtered prompt sound is attenuated to reduce the volume and weaken the high-frequency components. Finally, the processed audio is encoded into three-dimensional spatial audio data. The prompt sound heard by the user can not only be clearly perceived as coming from the front, but also the distance of the obstacle can be judged by the volume.

[0058] Furthermore, it should be noted that the embodiments of this application may not use the HRTF filter, but can use the binaural difference algorithm to simulate orientation. Three-dimensional positioning is achieved by calculating the time difference and intensity difference between the left and right ears. The accuracy is slightly lower (orientation error ±10°), but the computational efficiency is improved, making it suitable for low-computing-power cockpit platforms.

[0059] In step S103, audio middleware data is generated based on speaker data and three-dimensional spatial audio data, and the audio middleware data is rendered to generate vehicle alert sounds.

[0060] Among them, the audio middleware data is the audio output data adapted to the audio middleware.

[0061] It is understood that the embodiments of this application can generate audio middleware data based on speaker data and three-dimensional spatial audio data, and render the audio middleware data to generate vehicle prompts, making it easier for users to perceive the location of the target object and improving driving safety. Moreover, it does not rely on preset multi-scene and multi-directional audio files. By processing mono data and spatial state data in real time, it can flexibly generate prompts in various scenarios. At the same time, it can adjust the audio distribution according to the speaker data of different vehicle models, thus improving scalability.

[0062] Specifically, taking a static obstacle alarm 5m to the left rear as an example, with 6 speakers, this embodiment of the application can determine the core speaker and auxiliary speakers based on the position marker in the speaker data and the directional label in the three-dimensional spatial audio data. The core speaker is the left rear LR (directly matching the left rear direction), with a gain of +3dB (highlighting the main directional prompt); the auxiliary speaker is the left center LC (enhancing the sense of left rear direction layering), with a gain of -6dB; the suppressor speakers are the right front RF, right rear RR, and right headrest HR (to avoid directional confusion), with a gain of -20dB; the mute speakers are the left front LF, right center RC, and left headrest HL, with a gain of 0dB (no audio signal output).

[0063] Furthermore, based on the current vehicle cockpit system's audio middleware protocol (such as QNX's ALSA protocol and Android's AudioTrack protocol), the audio data with allocated channel gains is converted into a format recognizable by the middleware. Through standard APIs (such as the QNXsnd_pcm_writei function and Android's AudioTrack.write method), the audio middleware data is transmitted to the cockpit audio middleware. After receiving the data, the audio middleware calls the speaker hardware driver and controls the corresponding speaker output according to the channel number and gain parameters in the middleware data.

[0064] In one embodiment of this application, after rendering the audio middleware data, the method further includes: identifying the rendering result; if the rendering result is a rendering failure, then outputting a mono prompt corresponding to the mono prompt data.

[0065] It is understood that, in this embodiment of the application, after rendering the audio middleware data fails, a mono prompt tone corresponding to the mono prompt tone data is output to avoid the user being unable to obtain key warning information due to rendering failure, thereby improving driving safety.

[0066] This application embodiment also constructs a panoramic sound alert sound generation engine, which can efficiently and accurately generate alarm sounds or alert sounds with a sense of direction in real time. When there is a need for alarm sounds or alert sounds at the application layer, information such as the original mono file and sound location metadata can be passed in. The engine will automatically render the target alert sound based on this information and automatically write it to the audio middleware. At the same time, for safety alarm sounds, this engine also supports pre-rendering. When the cockpit system is powered on for the first time, it will automatically generate the necessary 3D alarm sounds based on the vehicle's speaker configuration to ensure the reliability of the safety system.

[0067] The main architecture of the panoramic sound prompt generation engine is as follows: Figure 3 As shown, it can achieve real-time rendering, adaptive adaptation, and secure preloading, including: 1. Immersive sound algorithm module.

[0068] Based on HRTF and vehicle acoustic environment calibration, the mono original audio (i.e. mono prompt tone data) is converted in real time into a multi-channel signal with three-dimensional spatial positioning (i.e. three-dimensional spatial audio data), supporting precise parameter input of azimuth (-180°~180°), elevation (-90°~90°) and distance (0~10m).

[0069] 2. Speaker layout adaptation unit.

[0070] It automatically identifies the number, location, and channel configuration of the vehicle speakers (such as an 8-speaker basic layout or a 23-speaker surround layout), dynamically adjusts the audio signal distribution strategy, and regenerates the pre-made files.

[0071] 3. Dynamic trajectory rendering engine.

[0072] For moving sound sources (such as a vehicle rapidly approaching from the left), the orientation parameters are updated in real time through an interpolation algorithm to achieve a smooth transition of sound from the rear left to the front left, simulating a natural spatial movement trajectory.

[0073] 4. Secure pre-rendering unit.

[0074] When the cockpit system is powered on for the first time, the engine automatically reads the vehicle's speaker configuration, pre-renders all safety warning sounds (such as reversing alarm and collision warning) and stores them in a high-speed cache to ensure that they play without delay when triggered.

[0075] 5. Application layer interface.

[0076] It provides a standardized API that allows the application layer to pass in the original mono audio file, location metadata (including static coordinates or dynamic trajectory parameters), and priority identifier (i.e., prompt generation identifier). The engine automatically completes the rendering and outputs it to the audio middleware.

[0077] The following will describe in detail the prompt sound generation method of this application embodiment in conjunction with the above-mentioned panoramic sound prompt sound generation engine.

[0078] 1. System deployment and initialization.

[0079] The panoramic sound alert generation engine (hereinafter referred to as the engine) is deployed on the vehicle's cockpit domain controller, developed based on the in-vehicle Linux or QNX system, and interacts with cockpit audio middleware (such as ALSA and AudioManager) through API interfaces. The initialization process is as follows: When the cockpit system is powered on for the first time, the engine automatically starts and triggers speaker layout detection: the speaker hardware information (number, position coordinates, channel mapping relationship) is obtained through the audio middleware. For example, a certain model is equipped with 8 speakers, distributed as front left (LF), front right (RF), rear left (LR), rear right (RR), center left (LC), center right (RC), left headrest (HL), and right headrest (HR). The coordinates are established in a three-dimensional coordinate system with the driver's seat as the origin (X-axis front and rear, Y-axis left and right, Z-axis up and down). The engine stores the detection results as a speaker profile for channel allocation in subsequent rendering.

[0080] 2. Real-time rendering.

[0081] Taking the left rear obstacle alarm scenario as an example, the real-time rendering process is explained in detail, such as... Figure 4 As shown.

[0082] S1: Receive input parameters.

[0083] Application layers (such as ADAS systems) pass in the following via API: Raw audio: mono alarm tone file (WAV format, 44.1kHz sampling rate); Location metadata: static orientation (azimuth - 60°, elevation 0°, distance 5m); Priority identifier: second identifier. S2: Immersive sound algorithm processing.

[0084] The panoramic sound algorithm module performs the following operations: HRTF Mapping: Based on the azimuth angle -60°, a preset HRTF filter is invoked to filter the mono audio, simulating the human ear's perception characteristics of sound from the left rear direction; Distance attenuation processing: Based on a distance of 5m, an attenuation curve is applied (attenuation coefficient = 1 / (distance)). 2 (), reduce the volume and increase high-frequency attenuation to simulate far-field sound characteristics; 3D spatial encoding: Encodes the processed audio signal into a 3D spatial audio stream (including orientation information tags). S3: Speaker layout adaptation.

[0085] The speaker layout adapter distributes the three-dimensional spatial audio stream to the corresponding channels according to the speaker configuration file: Left rear (LR) speaker: gain +3dB (main channel); Left center (LC) speaker: gain -6dB (auxiliary channel, enhances the sense of direction); Right front (RF) and right rear (RR) speakers: gain -20dB (suppression, avoids confusion about direction); other speakers: gain 0dB (mute).

[0086] S4: Output to audio middleware.

[0087] Depending on the deployed system, the corresponding audio output interface is called, such as Android AudioTrack or QNX's ALSA interface.

[0088] S5: Feedback the rendering results to the application layer.

[0089] The system will provide feedback to the requesting application indicating whether rendering was successful or not.

[0090] S6.: Use the in-vehicle microphone to check the actual sound output.

[0091] If successful, no special handling is required; if it fails to produce sound or the sound position is incorrect, a fault code is reported, and the backup original solution is used to produce sound.

[0092] 3. Dynamic trajectory rendering.

[0093] Taking the scenario of a vehicle rapidly overtaking on the left as an example, the dynamic trajectory is as follows: Figure 5 As shown.

[0094] Input parameters: Original audio: Mono prompt tone; Dynamic trajectory metadata: Starting orientation (-70°, 0°, 8m), ending orientation (30°, 0°, 3m), duration 2s; Motion mode: Uniform acceleration (speed increases from 10km / h to 30km / h). The processing includes: the dynamic trajectory rendering engine divides the 2-second time axis into 100 sampling points (once every 20ms), and calculates the azimuth angle (-70°→30°) and distance (8m→3m) of each sampling point through linear interpolation; each sampling point performs real-time rendering (same as steps S2-S3), and processes the gain difference between adjacent sampling points through a smoothing filtering algorithm (to avoid jumps); the output audio signal dynamically adjusts the gain of each speaker over time to achieve a continuous sound movement effect from the left rear to the left front, allowing the user to perceive the sound source accelerating towards the user.

[0095] 4. Secure pre-rendering mechanism.

[0096] When the cockpit system is powered on for the first time, perform the following operations, such as Figure 2 As shown.

[0097] Configuration Read: Reads the vehicle's speaker layout (e.g., 6 speakers) and safety warning tone list (5 categories including reversing alarm and collision warning); Batch Rendering: The panoramic sound engine pre-renders audio data for each type of safety warning tone based on all possible directional parameters (e.g., reversing alarm covers -90° to 90° directional range) and stores it in DRAM cache.

[0098] Verification and Update: Each time the cockpit system starts, the integrity of the pre-rendered data is verified. If the speaker configuration is changed (such as replacing the speakers after maintenance), the pre-rendering is automatically re-rendered.

[0099] Triggered playback: When the ADAS system triggers a safety prompt, it directly reads the pre-rendered data from the cache and outputs it through the audio middleware, with a response latency of ≤8ms.

[0100] In summary, the prompt sound generation method of this application can achieve the following effects: Significantly enhanced scalability: It can automatically generate pre-made audio files and adapt to any speaker layout through real-time rendering. Adding a new scene only requires passing parameters to generate the corresponding prompt sound, reducing adaptation costs by more than 80%; Enhanced immersion and dynamic accuracy: Based on panoramic sound algorithm, the 3D positioning has an orientation error controlled within ±5°, and the smoothness of dynamic trajectory transition is improved by more than 3 times, allowing users to accurately perceive the movement direction and speed of the sound source; Balanced safety and efficiency: The safety prompt sound pre-rendering mechanism ensures a response latency of ≤10ms, meeting the functional safety requirements of vehicles; at the same time, the real-time rendering engine adopts a lightweight algorithm with a CPU utilization of ≤5%, without affecting other functions of the cockpit system; Comprehensive scene coverage: It supports diverse scenes such as static orientation (e.g., left rear obstacle), dynamic trajectory (e.g., overtaking prompt), and area range (e.g., width simulation of multiple obstacles on the left), solving the scene limitations of traditional surround sound technology. The prompt sound generation method proposed in this application can encode mono prompt sound data into three-dimensional spatial audio data based on the spatial state data of targets around the vehicle, giving the audio a sense of direction and distance. Furthermore, it generates audio middleware data based on the vehicle's speaker data and three-dimensional spatial audio data, and renders the audio middleware data to generate the vehicle's prompt sound. It does not rely on preset multi-scene, multi-directional audio files. By processing mono data and spatial state data in real time, it can flexibly generate prompt sounds in various scenarios, and can adjust the audio distribution according to the speaker data of different vehicle models, thus improving scalability.

[0101] Next, the prompt sound generation apparatus according to the embodiments of this application is described with reference to the accompanying drawings.

[0102] Figure 6 This is a block diagram of a prompt sound generation device according to an embodiment of this application.

[0103] like Figure 6 As shown, the prompt sound generation device 10 includes: an acquisition module 100, an encoding module 200, and a generation module 300.

[0104] The acquisition module 100 is used to acquire the mono prompt sound data of the vehicle and the spatial state data of the target objects around the vehicle; the encoding module 200 is used to encode the mono prompt sound data into three-dimensional spatial audio data according to the spatial state data of the target objects, and acquire the speaker data of the vehicle; the generation module 300 is used to generate audio middleware data based on the speaker data and the three-dimensional spatial audio data, and render the audio middleware data to generate the vehicle's prompt sound.

[0105] In one embodiment of this application, the spatial state data includes at least one of orientation data, motion duration, and motion pattern.

[0106] In one embodiment of this application, the encoding module 200 is further configured to: identify the type of the target object; if the type of the target object is a first type, divide the motion duration into multiple sampling points, determine the target interpolation method based on the motion pattern, calculate the orientation data of each sampling point based on the target interpolation method, and encode the mono prompt sound data into three-dimensional spatial audio data based on the orientation data; if the type of the target object is a second type, encode the mono prompt sound data into three-dimensional spatial audio data based on the orientation data.

[0107] In one embodiment of this application, the apparatus 10 of this application embodiment further includes: a construction module.

[0108] The construction module is used to acquire the vehicle's mono prompt data and the spatial state data of the target objects around the vehicle before acquiring the vehicle's target coordinate system. The target coordinate system is constructed with the driver's seat as the origin, the front of the driver's seat as the positive X-axis, the right side of the driver's seat as the positive Y-axis, and the plane perpendicular to the X-axis and Y-axis as the positive Z-axis. The spatial state data of the target objects are determined based on the target coordinate system.

[0109] In one embodiment of this application, the orientation data includes at least one of azimuth, elevation, and distance.

[0110] In one embodiment of this application, the encoding module 200 is further configured to: invoke the corresponding head-related transfer function filter according to the azimuth angle and / or elevation angle, the head-related transfer function filter performing filtering processing on the mono prompt data; perform attenuation processing on the filtered mono prompt data based on the distance and target attenuation curves; and encode the attenuated mono prompt data into three-dimensional spatial audio data.

[0111] In one embodiment of this application, the apparatus 10 of this application embodiment further includes: an output module.

[0112] The output module is used to identify the rendering result after rendering the audio middleware data; if the rendering result is a rendering failure, it outputs the mono prompt tone corresponding to the mono prompt tone data.

[0113] In one embodiment of this application, the apparatus 10 of this embodiment further includes a rendering module.

[0114] The rendering module is used to identify the vehicle's prompt sound generation identifier before acquiring the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle. If the prompt sound generation identifier is the first identifier, the pre-rendered audio middleware data stored in the vehicle is acquired and rendered to generate the vehicle's prompt sound. Otherwise, the vehicle's mono prompt sound data and the spatial state data of the target objects around the vehicle are acquired.

[0115] In one embodiment of this application, the rendering module is further configured to: identify multiple mono prompt tone data in the vehicle's preset safety prompt tone list when the vehicle is started; obtain multiple directional data covered by each mono prompt tone data; and pre-render each mono prompt tone data based on the speaker data and the multiple directional data corresponding to each mono prompt tone to obtain multiple pre-rendered audio middleware data.

[0116] It should be noted that the foregoing explanation of the prompt sound generation method embodiment also applies to the prompt sound generation device of this embodiment, and will not be repeated here.

[0117] The prompt sound generation device proposed in the embodiments of this application can encode mono prompt sound data into three-dimensional spatial audio data based on the spatial state data of targets around the vehicle, giving the audio a sense of direction and distance. It further generates audio middleware data based on the vehicle's speaker data and three-dimensional spatial audio data, and renders the audio middleware data to generate the vehicle's prompt sound. It does not rely on preset multi-scene, multi-directional audio files. By processing mono data and spatial state data in real time, it can flexibly generate prompt sounds in various scenarios. Moreover, it can adjust the audio distribution according to the speaker data of different vehicle models, thus improving scalability.

[0118] Figure 7 A schematic diagram of the structure of a vehicle provided in an embodiment of this application. The vehicle may include: The memory 701, the processor 702, and the computer program stored on the memory 701 and executable on the processor 702.

[0119] When the processor 702 executes the program, it implements the prompt sound generation method provided in the above embodiments.

[0120] Furthermore, the vehicle also includes: Communication interface 703 is used for communication between memory 701 and processor 702.

[0121] The memory 701 is used to store computer programs that can run on the processor 702.

[0122] The memory 701 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0123] If the memory 701, processor 702, and communication interface 703 are implemented independently, then the communication interface 703, memory 701, and processor 702 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized into address buses, data buses, control buses, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0124] Optionally, in a specific implementation, if the memory 701, processor 702, and communication interface 703 are integrated on a single chip, then the memory 701, processor 702, and communication interface 703 can communicate with each other through an internal interface.

[0125] The processor 702 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0126] This application also provides a computer-readable storage medium storing a computer program or instructions thereon, which, when executed by a processor, implements the above-described prompt sound generation method.

[0127] This application also provides a computer program product, including a computer program or instructions, which, when executed, implement the above-described prompt sound generation method.

[0128] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0129] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0130] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0131] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or more of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0132] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A method for generating a prompt sound, characterized in that, Includes the following steps: Acquire the vehicle's mono audio prompt data and the spatial state data of targets around the vehicle; Based on the spatial state data of the target object, the monophonic prompt data is encoded into three-dimensional spatial audio data, and the speaker data of the vehicle is obtained. Audio middleware data is generated based on the speaker data and the three-dimensional spatial audio data, and the audio middleware data is rendered to generate the vehicle's prompt sound.

2. The prompt sound generation method according to claim 1, characterized in that, The spatial state data includes at least one of orientation data, movement duration, and movement pattern. Encoding the monophonic prompt data into three-dimensional spatial audio data based on the spatial state data of the target object includes: Identify the type of the target object; If the target object is of type 1, the motion duration is divided into multiple sampling points, and a target interpolation method is determined based on the motion mode. The orientation data of each sampling point is calculated based on the target interpolation method, and the mono prompt sound data is encoded into three-dimensional spatial audio data according to the orientation data. If the target object is of type two, then the mono prompt data is encoded into three-dimensional spatial audio data based on the orientation data.

3. The prompt sound generation method according to claim 1, characterized in that, Before acquiring the vehicle's mono audio prompt data and the spatial state data of targets around the vehicle, the process also includes: Obtain the target coordinate system of the vehicle, wherein the target coordinate system is constructed with the driver's seat of the vehicle as the origin, the front of the driver's seat as the positive X-axis, the right side of the driver's seat as the positive Y-axis, and the positive Z-axis perpendicular to the plane formed by the X-axis and the Y-axis. The spatial state data of the target object are determined based on the target coordinate system.

4. The prompt sound generation method according to claim 2, characterized in that, The azimuth data includes at least one of azimuth angle, elevation angle, and distance. Encoding the mono tone data into three-dimensional spatial audio data based on the azimuth data includes: The corresponding head-related transfer function filter is invoked according to the azimuth angle and / or elevation angle, and the head-related transfer function filter performs filtering processing on the mono prompt tone data; The filtered monophonic prompt data is attenuated based on the distance and target attenuation curve. The attenuated mono prompt data is encoded into three-dimensional spatial audio data.

5. The prompt sound generation method according to claim 1, characterized in that, After rendering the audio middleware data, the following is also included: Identify the rendering result; If the rendering result is a rendering failure, then the mono prompt tone corresponding to the mono prompt tone data is output.

6. The prompt sound generation method according to claim 1, characterized in that, Before acquiring the vehicle's mono audio prompt data and the spatial state data of targets around the vehicle, the process also includes: The system identifies the vehicle's prompt sound and generates an identifier; If the prompt sound generation identifier is the first identifier, then the pre-rendered audio middleware data stored in the vehicle is obtained, and the pre-rendered audio middleware data is rendered to generate the prompt sound of the vehicle; otherwise, the mono prompt sound data of the vehicle and the spatial state data of the target objects around the vehicle are obtained.

7. The prompt sound generation method according to claim 1, characterized in that, The step of obtaining the pre-rendered audio middleware data pre-stored in the vehicle includes: When the vehicle is started, multiple mono tone data from the vehicle's preset safety warning tone list are identified; Acquire multiple directional data covered by each mono tone data; Based on the speaker data and multiple directional data corresponding to each mono prompt tone, each mono prompt tone data is pre-rendered to obtain multiple pre-rendered audio middleware data.

8. A prompt sound generation device, characterized in that, include: The acquisition module is used to acquire the vehicle's mono audio prompt data and the spatial state data of the target objects around the vehicle; The encoding module is used to encode the mono prompt sound data into three-dimensional spatial audio data based on the spatial state data of the target object, and to acquire the speaker data of the vehicle. The generation module is used to generate audio middleware data based on the speaker data and the three-dimensional spatial audio data, and to render the audio middleware data to generate the vehicle's prompt sound.

9. A vehicle, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the prompt sound generation method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, The computer program or instructions are executed by a processor to implement the prompt sound generation method as described in any one of claims 1-7.

Images