Obstacle prompting method and vehicle-mounted audio system

Abstract

The invention provides an obstacle prompting method and a vehicle-mounted audio system, and the method comprises the steps: determining a target prompt tone corresponding to an obstacle from a prompt tone library based on data collected by a vehicle sensor, and determining the coordinates of the obstacle in a coordinate system of a vehicle; performing coordinate conversion on the obstacle, and converting the coordinate of the obstacle into a coordinate in a spherical coordinate system taking the listening position as an original point; and rendering the target prompt tone by taking the coordinate of the obstacle in the spherical coordinate system as a sound source coordinate. Proper prompt tones are made for the obstacles according to the data of the obstacles collected by the sensors, so that the accuracy of the prompt tones is improved, and the content and interestingness of the prompt tones are enriched; and the spherical coordinates of the obstacle are taken as the sound source coordinates to carry out prompt tone rendering, so that the effect of prompting the direction of the obstacle through the prompt tone is achieved, the obstacle prompting effect is improved, and the driving safety is improved.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and in particular to an obstacle prompting method and a vehicle-mounted audio system. Background Art

[0002] With the continuous advancement of vehicle-mounted sensor technology, the types of vehicle-mounted sensors are becoming more and more abundant, and the accuracy of sensors is becoming higher and higher, which can sense obstacles around the vehicle and provide obstacle warnings.

[0003] In vehicle warning technology, the warning sound is usually monophonic audio data, and the warning content is fixed, such as a simple beeping sound. The warning method is monotonous and cannot play a sufficient warning role, resulting in poor driving safety.

[0004] Therefore, there is an urgent need to provide a rich obstacle warning solution with good warning effect to improve driving safety. Summary of the invention

[0005] The present application provides an obstacle warning method and an in-vehicle audio system, which improves the warning effect and driving safety by modeling the sound source of the obstacle and rendering a warning sound that shows the position of the obstacle. At the same time, the warning content that matches the obstacle is used for warning, which improves the richness and fun of the warning.

[0006] In a first aspect, the present application provides an obstacle prompting method, comprising:

[0007] Based on the data collected by the vehicle sensor, determine the target prompt sound corresponding to the obstacle from the prompt sound library, and determine the coordinates of the obstacle in the coordinate system of the vehicle;

[0008] Performing coordinate transformation on the obstacle, transforming the coordinates of the obstacle into coordinates in a spherical coordinate system with the listening position as the origin;

[0009] The target prompt sound is rendered using the coordinates of the obstacle in the spherical coordinate system as the sound source coordinates.

[0010] In some embodiments, based on data collected by vehicle sensors, determining a target prompt sound corresponding to the obstacle from a prompt sound library includes:

[0011] Determine attribute information of the obstacle based on data collected by the vehicle sensor, where the attribute information is used to describe at least one of the type of the obstacle, the moving speed, and the distance from the vehicle;

[0012] Based on the attribute information of the obstacle, determining a prompt scene corresponding to the obstacle;

[0013] The prompt sound corresponding to the prompt scene is determined from the prompt sound library as the target prompt sound corresponding to the obstacle.

[0014] In some embodiments, based on the attribute information of the obstacle, determining the prompt scene corresponding to the obstacle includes:

[0015] If the type of the obstacle is a pedestrian and the distance between the obstacle and the vehicle is greater than a first distance, determining that the prompt sound scene corresponding to the obstacle is a far-field pedestrian scene;

[0016] If the type of the obstacle is a pedestrian and the distance between the obstacle and the vehicle is less than or equal to the first distance, the prompt scene corresponding to the obstacle is a near-field pedestrian scene.

[0017] In some embodiments, based on the attribute information of the obstacle, determining the prompt scene corresponding to the obstacle includes:

[0018] If the distance between the obstacle and the vehicle is greater than or equal to a preset distance, or if the distance between the obstacle and the vehicle is less than the preset distance, the moving speed of the obstacle relative to the vehicle is less than or equal to a set speed, and the distance between the obstacle and the vehicle is less than or equal to a safe braking distance at the moving speed of the obstacle relative to the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a stationary prompt scene;

[0019] If the distance between the obstacle and the vehicle is less than a preset distance, and the distance between the obstacle and the vehicle is greater than a safe braking distance of the obstacle relative to the moving speed of the vehicle, determining that the warning sound scene corresponding to the obstacle is a dangerous movement warning scene;

[0020] If the distance between the obstacle and the vehicle is less than a preset distance, the moving speed of the obstacle relative to the vehicle is greater than a set speed, and the distance between the obstacle and the vehicle is less than or equal to the safe braking distance of the obstacle at the moving speed of the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a high-speed moving prompt scene.

[0021] In some embodiments, the coordinates of the obstacle in the spherical coordinate system include the coordinates of multiple sampling points on the obstacle in the spherical coordinate system, and the coordinates of the obstacle in the spherical coordinate system are used as the sound source coordinates, and rendering the target prompt sound includes:

[0022] The target prompt sound is rendered based on a preset rendering algorithm, taking each of the sampling points as a point sound source and taking the coordinates of each of the sampling points in the spherical coordinate system as point sound source coordinates.

[0023] In some embodiments, performing coordinate transformation on the obstacle to transform the coordinates of the obstacle into coordinates in a spherical coordinate system with the listening position as the origin includes:

[0024] Establishing an obstacle model of the obstacle based on coordinates of a plurality of sampling points on the obstacle in a coordinate system of the vehicle;

[0025] The obstacle model is subjected to coordinate transformation to obtain the obstacle model in a spherical coordinate system with the listening position as the origin.

[0026] Correspondingly, using the coordinates of the obstacle in the spherical coordinate system as the sound source coordinates, rendering the target prompt sound includes:

[0027] The obstacle model after coordinate transformation is used as the sound source model, and the target prompt sound is rendered based on a preset rendering algorithm.

[0028] In some embodiments, the method further comprises:

[0029] Determining whether the moving speed of the obstacle is less than a preset speed;

[0030] If yes, determining that the preset rendering algorithm is the first rendering algorithm;

[0031] If not, determining that the preset rendering algorithm is the second rendering algorithm;

[0032] Among them, the first rendering algorithm is a sound bed rendering algorithm or a panoramic sound rendering algorithm; the second rendering algorithm is a panning algorithm or a wave field synthesis rendering algorithm.

[0033] In some embodiments, the method further comprises:

[0034] Get the listening position specified by the user; or,

[0035] The listening position is determined based on a position of a user sitting in the vehicle.

[0036] In some embodiments, the method further comprises:

[0037] When a plurality of users are seated in the vehicle, the listening position is determined based on the seat position of each of the users, the position priority of the seat position of each of the users, and the priority of each of the users.

[0038] In a second aspect, the present application also provides an obstacle prompting device, comprising:

[0039] A target prompt sound determination module is used to determine the target prompt sound corresponding to the obstacle from the prompt sound library based on the data collected by the vehicle sensor, and determine the coordinates of the obstacle in the coordinate system of the vehicle;

[0040] A coordinate conversion module, used for performing coordinate conversion on the obstacle, converting the coordinates of the obstacle into coordinates in a spherical coordinate system with the listening position as the origin;

[0041] The prompt sound rendering module is used to render the target prompt sound by taking the coordinates of the obstacle in the spherical coordinate system as the sound source coordinates.

[0042] In a third aspect, the present application further provides a vehicle audio system, which is arranged in a vehicle and includes:

[0043] A sensor and a renderer; the sensor is used to collect data on obstacles outside the vehicle; the obstacle prompt sound generating device is used to execute the obstacle prompt method provided in any embodiment of the present application to generate the obstacle prompt sound; the renderer is used to render the prompt sound so as to play the prompt sound through the speaker array of the vehicle.

[0044] In a fourth aspect, the present application also provides an electronic device, comprising a memory and at least one processor, wherein the memory stores computer-executable instructions; the at least one processor executes the computer-executable instructions stored in the memory, so that the electronic device implements the method provided in the first aspect.

[0045] In a fifth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the method provided in the first aspect is implemented.

[0046] In a sixth aspect, an embodiment of the present application further provides a computer program product, including a computer program, which implements the method provided in the first aspect when executed by a processor.

[0047] The obstacle warning method and vehicle audio system provided by the present application are aimed at application scenarios of warning of obstacles outside the vehicle. Based on the obstacle data collected by the vehicle sensor, a warning sound is personalized for the obstacle, thereby improving the accuracy and content richness of the obstacle warning sound. When rendering the obstacle warning sound, in order to facilitate rendering, the coordinates of the obstacle are first transformed, and the coordinates of the obstacle are converted into spherical coordinates. The spherical coordinates of the obstacle are used as the sound source coordinates of the warning sound, and the obstacle warning sound is rendered, so as to render a warning sound that can indicate the direction of the obstacle, thereby improving the effect of the obstacle warning, reducing the possibility of collision between the vehicle and the obstacle, and improving the safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0049] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application;

[0050] Figure 2 A flowchart of an obstacle prompting method provided in an embodiment of the present application;

[0051] Figure 3 A flowchart of another obstacle prompting method provided in an embodiment of the present application;

[0052] Figure 4 The prompt scenario provided for one embodiment of the present application is divided into a schematic diagram;

[0053] Figure 5 A schematic diagram of a target prompt sound rendering effect provided in an embodiment of the present application;

[0054] Figure 6 A flowchart of another obstacle prompting method provided in an embodiment of the present application;

[0055] Figure 7 A schematic diagram of the structure of a vehicle audio system provided in an embodiment of the present application;

[0056] Figure 8 A schematic diagram of the structure of an obstacle warning system provided in an embodiment of the present application.

[0057] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0058] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0059] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application, such as Figure 1 As shown, the vehicle is equipped with an ADAS (Advanced Driving Assistance System). After the vehicle is started, the ADAS uses installed sensors, such as infrared sensors, cameras, radars, etc., to sense the environment around the vehicle, collect environmental data, and detect obstacles.

[0060] After ADAS detects an obstacle, Figure 1 The direction of the pedestrians, vehicles and pedestrians is as follows: Figure 1 As shown, if there is a risk of collision between the vehicle and pedestrians, a warning can be issued through the speakers on the vehicle, such as controlling the central speaker to emit a warning sound, such as a short beep, to remind the driver to pay attention to pedestrians outside the vehicle.

[0061] In the related art, the obstacle warning method is single, and the driver can only determine the existence of obstacles around the vehicle through the monophonic prompt sound (such as beep) emitted by the speaker. The driver needs to further determine the position of the obstacle, the distance from the vehicle and other information in combination with the rear mirror and reversing image, which is not conducive to driving safety. Especially in the environment where the vehicle is located with low visibility, the driver will not be able to confirm the location of the obstacle, and it is easy to collide with the obstacle, resulting in poor driving safety.

[0062] With the continuous development of panoramic sound technology, the speaker channels on vehicles are becoming more and more abundant. In order to improve the effect of obstacle prompts, the present application provides an obstacle prompt method, which uses the data collected by the vehicle sensor to realize differentiated matching of obstacle prompt sounds (i.e., target prompt sounds), thereby improving the accuracy and richness of the prompt sounds; using the spherical coordinates of the obstacle as the sound source coordinates, the obstacle prompt sound is rendered to render a prompt sound that indicates the obstacle's position, thereby improving the effect of obstacle prompts and driving safety.

[0063] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0064] Figure 2 This is a flow chart of an obstacle prompting method provided in an embodiment of the present application. The method is applied to a vehicle and can be executed by a renderer or obstacle prompting device or system on the vehicle, or by a server or other remote device. Figure 2 As shown, the obstacle prompting method comprises the following steps:

[0065] Step S201, based on the data collected by the vehicle sensor, determine the target prompt sound corresponding to the obstacle from the prompt sound library, and determine the coordinates of the obstacle in the coordinate system of the vehicle.

[0066] The vehicle coordinate system may be a coordinate system with the left front wheel of the vehicle as the origin and the driving direction of the vehicle as the positive direction of the longitudinal axis.

[0067] Sensors on the vehicle include but are not limited to infrared sensors, image sensors, millimeter wave radar sensors, lidar sensors, ultrasonic sensors, etc.

[0068] The vehicle is provided with a speaker array, and the number of speakers in the speaker array is at least 3.

[0069] During the driving process of the vehicle, the sensors installed on the vehicle collect data about the surrounding environment. When it is sensed that there is an obstacle in the vehicle's surrounding environment, the obstacle data collected by the sensor is obtained, and based on the obstacle data, a prompt sound is matched for the obstacle from the prompt sound library to obtain the target prompt sound corresponding to the obstacle.

[0070] Obstacles can be stationary objects, such as traffic lights, guardrails, trees, walls, water barriers, etc., or moving objects, such as pedestrians, other vehicles, etc. Obstacles can be sound-emitting objects or silent objects.

[0071] Specifically, the attribute information of the obstacle may be determined based on the data of the obstacle collected by the sensor, and based on the attribute information of the obstacle, the target prompt sound corresponding to the obstacle may be determined from the prompt sound library.

[0072] The attribute information includes but is not limited to the type, size, material, temperature, moving speed, position relative to the vehicle, distance from the vehicle, and sound absorption coefficient of the obstacle.

[0073] The obstacle's size, distance from the obstacle to the vehicle and other attribute information can be determined based on obstacle data collected by a millimeter-wave radar sensor or a lidar sensor installed on the vehicle.

[0074] At night or in other low-visibility environments, the size of the obstacle, the distance between the obstacle and the vehicle, the temperature of the obstacle and other attribute information can be determined based on the data collected by the infrared sensor installed on the vehicle.

[0075] Based on the image of the obstacle captured by the image sensor installed on the vehicle, such as a monocular / binocular camera, the obstacle type, obstacle size, distance between the obstacle and the vehicle, obstacle material and other attribute information can be identified.

[0076] Based on the obstacle data collected by the ultrasonic sensor installed on the vehicle, attribute information such as the distance between the obstacle and the vehicle and the sound absorption coefficient of the obstacle can be determined.

[0077] In some embodiments, the category of the obstacle, such as pedestrians, dogs, red lights, walls, etc., can also be determined based on the audio data of the obstacle.

[0078] In other embodiments, an obstacle scene may be determined based on the attribute information of the obstacle, and based on the obstacle scene, a target prompt sound corresponding to the obstacle may be determined from a prompt sound library.

[0079] Based on the attribute information of various possible obstacles, multiple obstacle scenes can be divided in advance, and the scene conditions that the attribute information should meet in each obstacle scene can be summarized. The prompt sound set for each obstacle scene is stored in the prompt sound library, and the corresponding obstacle scene label is added to the prompt sound. The target prompt sound corresponding to the obstacle can be determined from the prompt sound library through the prompt sound label and the determined obstacle scene.

[0080] In some embodiments, data collected by sensors installed on the vehicle can be obtained at a certain period, and based on the data collected during the period, it can be determined whether there is an obstacle; if so, the target prompt sound corresponding to the obstacle is determined from the prompt sound library based on the data collected during the period, and the coordinates of the obstacle in the vehicle's coordinate system are determined. The period can be a fixed period, such as 1s, or a period determined based on the vehicle speed. The faster the vehicle speed, the shorter the period.

[0081] Step S202: performing coordinate transformation on the obstacle, and transforming the coordinates of the obstacle into coordinates in a spherical coordinate system with the listening position as the origin.

[0082] The listening position is a position on the vehicle, which may be the driver's head or the head of a passenger in the vehicle. The listening position may be manually specified, such as being the position of the driver's head by default, or may be determined based on the position of the passenger in the vehicle detected.

[0083] The listening position may be determined based on a selection instruction input by the user.

[0084] In some embodiments, the selection instructions may include the spherical coordinates of the listening position.

[0085] In other embodiments, the selection instruction may include a selected seat, such as the vehicle's main driver's seat, the co-pilot seat, the left rear seat, etc., and the listening position is determined based on the selected seat.

[0086] The user can input a selection instruction through the interactive device. After receiving the selection instruction, the renderer on the vehicle determines the listening position based on the selection instruction and establishes a spherical coordinate system with the listening position as the origin.

[0087] In some embodiments, the selection instruction can be sent to a remote device, such as a server, a user terminal, etc., through the vehicle's networking device. The remote device determines the listening position based on the selection instruction and establishes a spherical coordinate system with the listening position as the origin.

[0088] When there are multiple users in the vehicle, the coordinates of the center of the head of any one of the users can be used as the coordinates of the target listening position, such as the user who frequently rides the vehicle, the user with the highest priority, etc.

[0089] The listening position may also be the position of the head of the identified user. When there are multiple users, a target user may be selected from the multiple users, and the head of the target user may be used as the listening position. The listening position may also be calculated based on the positions of the heads of multiple users, such as the listening position being the average of the positions of the heads of multiple users. The listening position may also be determined based on the user level and the positions of the heads of multiple users, specifically by determining the weight based on the user level, and using the weighted average of the positions of the heads of multiple users as the listening position.

[0090] After establishing a spherical coordinate system with the listening position as the origin, based on the conversion relationship between the spherical coordinate system and the vehicle's coordinate system, the coordinates of the obstacle in the vehicle's coordinate system are transformed to obtain the spherical coordinates of the obstacle, that is, the coordinates in the spherical coordinate system.

[0091] Step S203: Render the target prompt sound using the coordinates of the obstacle in the spherical coordinate system as the sound source coordinates.

[0092] The spherical coordinates of the obstacle are used as the sound source coordinates, and the target prompt sound is rendered so that the rendered target prompt sound heard by the listener at the listening position is the sound emitted from the sound source coordinates, thereby achieving the effect of identifying the position by sound.

[0093] In some embodiments, a panoramic sound rendering algorithm, such as a high-order panoramic sound rendering algorithm, may be used to render the target prompt sound to improve the spatial effect of the rendering.

[0094] Specifically, based on the channel layout, position layout and listening position of the speakers in the vehicle, the spherical coordinates of the obstacle can be used as the sound source coordinates to render the target prompt sound corresponding to the obstacle, so that the rendering effect matches the layout of the speakers in the vehicle and the position of the listening position, thereby optimizing the auditory effect of the listening position.

[0095] Specifically, based on the coordinates of the listening position and the speakers in the vehicle, a synthetic sound intensity function of the speakers in the vehicle at the listening position can be determined, and the synthetic sound intensity function is a function that describes the synthetic sound intensity of each speaker in the vehicle with respect to the gain of each speaker channel in the vehicle when rendering the target prompt sound. Based on the virtual sound source corresponding to the target prompt sound, the virtual sound intensity of the virtual sound source at the target listening position is determined, and the virtual sound intensity is reconstructed using the synthetic sound intensity function to solve the gain of each speaker channel in the vehicle when rendering the target prompt sound.

[0096] Based on the Green's function, the sound pressure function of the virtual sound source at the target prompt sound source coordinates can be described, and combined with the acoustic Euler equation, the virtual sound intensity of the virtual sound source at the listening position can be obtained.

[0097] The composite sound intensity function is a function that describes the composite sound intensity of each speaker in the vehicle with respect to the gain of each speaker channel in the vehicle when rendering the audio signal to be rendered.

[0098] Specifically, the synthetic sound intensity function of each speaker in the vehicle at the target listening position can be determined based on the coordinates of the target listening position and the coordinates of the speakers in the vehicle, as well as parameters such as the orientation and frequency band of the speakers in the vehicle.

[0099] Specifically, the sound pressure of the speakers in the vehicle at the target listening position can be determined based on the coordinates of the target listening position, as well as the coordinates, frequency and orientation of the speakers in the vehicle; based on the sound pressure of each speaker in the vehicle at the target listening position, the synthetic sound intensity function of each speaker in the vehicle at the target listening position can be obtained.

[0100] The Green's Function can be used to describe the sound pressure of the sound source corresponding to each speaker in the vehicle.

[0101] The sound source corresponding to the loudspeaker can be regarded as a point sound source or a surface sound source.

[0102] Based on the distance of the loudspeaker relative to the target listening position, it can be determined whether the sound source corresponding to the loudspeaker is a point sound source or a surface sound source.

[0103] Exemplarily, when the distance between the speaker and the target listening position is less than or equal to a preset distance, the sound source corresponding to the speaker is determined to be a surface sound source; otherwise, the sound source corresponding to the speaker is determined to be a point sound source.

[0104] The preset distance may be a default value, or may be obtained based on the size of the speaker, such as a preset multiple of the size of the speaker, where the preset multiple may be 8 times, 10 times, 15 times, or other multiples. The size of the speaker may be a speaker diaphragm radius.

[0105] For a loudspeaker with a point sound source, the spherical wave of the point sound source corresponding to the loudspeaker is described by the Green function to obtain the sound pressure function of the loudspeaker.

[0106] For a speaker with a surface sound source, several points on the speaker grille can be taken as point sound sources, and the sound pressure of the several point sound sources corresponding to the speaker can be described by the Green function to obtain the spherical waves of the sound pressure of the several point sound sources, and the spherical waves of the several point sound sources can be superimposed to obtain the sound pressure function of the speaker with a surface sound source. In some embodiments, the synthetic sound intensity function includes a first part and a second part, the first part being a function of the active sound intensity synthesized by the sound pressure and vibration velocity generated by the speaker corresponding to the same speaker channel in the active sound intensity of the synthetic sound intensity, and the second part being a function of the active sound intensity generated by the coupling between speakers of different speaker channels in the active sound intensity of the synthetic sound intensity. The active sound intensity of the virtual sound intensity can be made equal to the above-mentioned first part, and the gain of each speaker channel in the vehicle when rendering the target prompt sound can be solved with the goal of minimizing the above-mentioned second part.

[0107] In the technical solution provided in this embodiment, for the application scenario of prompting obstacles outside the vehicle, based on the obstacle data collected by the vehicle sensor, a personalized prompt sound is matched for the obstacle, thereby improving the accuracy and content richness of the obstacle prompt sound; when rendering the obstacle prompt sound, in order to facilitate rendering, the obstacle coordinates are first transformed, the obstacle coordinates are converted into spherical coordinates, and the spherical coordinates of the obstacle are used as the sound source coordinates of the prompt sound, and the obstacle prompt sound is rendered to render a prompt sound that can prompt the direction of the obstacle, thereby improving the effect of the obstacle prompt, reducing the possibility of collision between the vehicle and the obstacle, and improving the safety of vehicle driving.

[0108] Figure 3 A flowchart of another obstacle prompting method provided in an embodiment of the present application. Figure 2 Based on the embodiment shown, step S201 and step S203 are further refined, and steps related to obtaining the listening position are added before step S202, such as Figure 3 As shown, the obstacle prompting method provided in this embodiment specifically includes the following steps:

[0109] Step S301: determining the attribute information of the obstacle and the coordinates of the obstacle in the coordinate system of the vehicle based on the data collected by the vehicle sensor.

[0110] The attribute information is used to describe at least one of the type of the obstacle, the moving speed, and the distance from the vehicle. The moving speed can be the moving speed of the obstacle relative to the ground or the moving speed relative to the vehicle. If not specifically specified, it means the moving speed relative to the ground.

[0111] The attribute information may also include the position of the obstacle relative to the vehicle, the size of the obstacle relative to the vehicle, the temperature of the obstacle, the sound absorption coefficient of the obstacle, etc. The temperature may be determined by the obstacle data collected by the infrared sensor, and the sound absorption coefficient may be determined by the echo signal of the obstacle received by the ultrasonic sensor.

[0112] Specifically, the category of the obstacle can be determined based on the obstacle image captured by the image sensor. When the image sensor is a 3D camera or a binocular camera, the three-dimensional coordinates of the obstacle in the vehicle coordinate system, such as (x, y, z), can also be determined based on the captured obstacle image. The vehicle coordinate system can be a coordinate system with the left front wheel of the vehicle as the origin and the driving direction of the vehicle as the positive direction of the longitudinal axis.

[0113] The distance between the obstacle and the vehicle can be determined based on data collected by one or more of an infrared sensor, a radar sensor, an ultrasonic sensor, etc.

[0114] After obtaining obstacle-related data collected by vehicle sensors, such as obstacle images, relevant recognition algorithms, such as image recognition algorithms, can be used to identify the category of the obstacle, its position relative to the vehicle, etc.; by processing the data collected by the sensor, such as clustering, the size or dimension of the obstacle, the position of the obstacle relative to the vehicle, and the moving speed or approaching speed of the obstacle can be determined.

[0115] Based on the ultrasonic waves returned by the obstacle collected by the ultrasonic sensor, the distance between the obstacle and the vehicle can be determined, and the sound absorption coefficient of the obstacle can also be determined.

[0116] Step S302: determining a prompt scene corresponding to the obstacle based on the attribute information of the obstacle.

[0117] The prompt scene corresponding to the obstacle can be determined based on the values ​​of multiple pieces of attribute information corresponding to the obstacle.

[0118] The conditions that the attribute information of the obstacle should meet in multiple prompting scenarios can be pre-configured to obtain the corresponding relationship between the prompting scenario and the attribute condition, where the attribute condition is the condition that the attribute information needs to meet. After obtaining the attribute information of the obstacle, the prompting scenario in which the attribute information of the obstacle meets the attribute condition is determined as the prompting scenario corresponding to the obstacle.

[0119] According to the moving speed, the prompt scene can be divided into a stationary object prompt scene and a moving object prompt scene. The moving speed of the obstacle corresponding to the stationary object prompt scene is 0, and the moving speed of the obstacle corresponding to the moving object prompt scene is greater than 0. The moving object prompt scene can be further divided into a fast-moving object prompt scene and a slow-moving object prompt scene according to the moving speed.

[0120] According to the type of obstacle, the warning scene can also be divided into pedestrian warning scene (the obstacle is a pedestrian), traffic light warning scene (the obstacle is a traffic light), vehicle warning scene (the obstacle is a vehicle, bicycle or other vehicle), etc.

[0121] According to the distance between the obstacle and the vehicle, the prompt scene can be divided into a near-view prompt scene and a far-view prompt scene. In the near-view prompt scene, the distance between the obstacle and the vehicle is less than or equal to the first distance, such as 2m, 1.5m, 1m or other values. In the far-view prompt scene, the distance between the obstacle and the vehicle is greater than the first distance.

[0122] The prompting scenes can be divided by combining multiple attribute information of obstacles, such as near-field moving object prompting scene, far-field moving pedestrian prompting scene, far-field traffic light prompting scene, etc. Near-field means that the distance between the obstacle and the vehicle is less than or equal to the first distance, far-field means that the distance between the obstacle and the vehicle is greater than the first distance, moving object means that the moving speed of the obstacle is not 0, and moving pedestrian means that the obstacle is a pedestrian and the moving speed of the pedestrian is not 0.

[0123] One or more thresholds may be set for the attribute information, and based on the comparison result between the attribute information and the corresponding thresholds, the prompt scene of the obstacle is determined.

[0124] The prompting scenarios may include traffic light prompting scenarios, pedestrian passing prompting scenarios, high-speed collision prompting scenarios, etc. In the traffic light prompting scenario, the obstacle type is a traffic light. In the pedestrian passing prompting scenario, the obstacle type is a person, and the obstacle moves at a low speed. In the high-speed collision prompting scenario, the obstacle moves at a high speed.

[0125] Optionally, determining a prompt scene corresponding to the obstacle based on the attribute information of the obstacle includes:

[0126] If the type of the obstacle is a pedestrian and the distance between the obstacle and the vehicle is greater than the first distance, the prompt sound scene corresponding to the obstacle is determined to be a far-field pedestrian scene; if the type of the obstacle is a pedestrian and the distance between the obstacle and the vehicle is less than or equal to the first distance, the prompt scene corresponding to the obstacle is a near-field pedestrian scene.

[0127] The first distance may be 2 m, 1.5 m, 1 m or other values.

[0128] Optionally, determining a prompt scene corresponding to the obstacle based on the attribute information of the obstacle includes:

[0129] If the distance between the obstacle and the vehicle is greater than or equal to a preset distance, or if the distance between the obstacle and the vehicle is less than the preset distance, the moving speed of the obstacle relative to the vehicle is less than or equal to the set speed, and the distance between the obstacle and the vehicle is less than or equal to the safe braking distance of the obstacle relative to the moving speed of the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a stationary prompt scene; if the distance between the obstacle and the vehicle is less than the preset distance, and the distance between the obstacle and the vehicle is greater than the safe braking distance of the obstacle relative to the moving speed of the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a dangerous high-speed moving prompt scene; if the distance between the obstacle and the vehicle is less than the preset distance, the moving speed of the obstacle relative to the vehicle is greater than the set speed, and the distance between the obstacle and the vehicle is less than or equal to the safe braking distance of the obstacle relative to the moving speed of the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a high-speed moving prompt scene.

[0130] The preset distance may be a relatively long distance, which is much greater than the first distance, and may be 100m, 150m, 200m, 300m, 320m or other values. The set speed may be 25Km / h, 30Km / h, 35Km / h or other values.

[0131] The safe braking distance of a vehicle when an obstacle is relative to the moving speed of the vehicle can be described by a safe braking curve. The safe braking curve is used to describe the maximum braking distance of the vehicle when each obstacle is relative to the moving speed of the vehicle. If the distance between the obstacle and the vehicle is less than the maximum braking distance, a collision is likely to occur.

[0132] The safe braking curve can be replaced by a safe braking equivalent curve, which is the ratio of the obstacle's moving speed relative to the vehicle to the maximum distance of the vehicle's safe braking at the moving speed. If the ratio of the obstacle's moving speed relative to the vehicle to the distance between the obstacle and the vehicle is greater than the ratio in the safe braking equivalent curve at the moving speed, a collision is likely to occur.

[0133] For example, Figure 4 The prompt scene provided for one embodiment of the present application is divided into a schematic diagram. Figure 4 In the prompt scene division, two attribute information, the distance L between the obstacle and the vehicle and the moving speed V of the obstacle relative to the vehicle, are mainly considered, such as Figure 4As shown in the figure, if the distance L between the obstacle and the vehicle is greater than the preset distance (for example, 320m), there is no need to consider the moving speed V of the obstacle relative to the vehicle, and the prompt scene corresponding to the obstacle is directly determined as the static prompt scene. When the value of L is less than or equal to the preset distance, it is necessary to comprehensively determine the prompt scene by combining the two factors of V and L. A value of V greater than 0 indicates that the obstacle is approaching the vehicle or the vehicle is approaching the obstacle.

[0134] Continue to see Figure 4 When the value of L is less than or equal to the preset distance, the following methods can be used to divide the prompt scene:

[0135] If the value of V is less than or equal to the set speed (such as 30Km / h), and L is less than or equal to the safe braking distance under the corresponding V in the safety braking curve, the prompt scene is determined to be a stationary prompt scene; if the value of V is greater than the set speed (such as 30Km / h), and L is less than or equal to the safe braking distance under the corresponding V in the safety braking curve, the prompt scene is determined to be a high-speed moving prompt scene; if the value of L is greater than the safe braking distance under the corresponding V in the safety braking curve, the prompt scene is determined to be a dangerous high-speed moving prompt scene.

[0136] In some embodiments, the set speed can be determined based on the speed corresponding to the preset distance in the safety braking curve, such as the ratio of the set speed to the speed corresponding to the preset distance in the safety braking curve is 1 / 10, 1 / 8, 1 / 5 or other values.

[0137] Furthermore, the above-mentioned prompting scenes may be further subdivided according to the types of obstacles.

[0138] By combining the prompt scene division with multiple attribute information, it is possible to reasonably divide the obstacle prompt scene into multiple scenes with high scene division accuracy, so that different scenes use different prompt sounds for prompts, which improves the fit between the prompt sound and the scene and improves the accuracy of the prompt sound.

[0139] Step S303: determining from a prompt sound library that the prompt sound corresponding to the prompt scene is the target prompt sound corresponding to the obstacle.

[0140] Based on the prompt scene corresponding to the obstacle, the prompt sound corresponding to the prompt scene is searched from the prompt sound library, such as searching by the prompt scene label, and the prompt sound corresponding to the prompt sound scene is used as the prompt sound of the obstacle, that is, the target prompt sound.

[0141] Exemplarily, the target prompt tone in the pedestrian prompt scenario may be "Watch out for pedestrians". The target prompt tone may also be determined in combination with the prompt scenario and the position of the pedestrian relative to the vehicle, such as "Watch out for pedestrians behind the left side of the vehicle".

[0142] Exemplarily, the target prompt sound in a mobile prompt scenario, such as a dangerous mobile prompt scenario, a high-speed mobile prompt scenario, etc., can use the sound emitted by a sound source model with a Doppler effect added. The prompt content of the target prompt sound in the dangerous mobile prompt scenario can be "Warning, obstacles are approaching quickly, please slow down."

[0143] For example, the prompt content of the target prompt sound in the traffic light prompt scenario can be further determined in combination with the signal recognition of red and green, such as "the red light is about to turn green", "the red light has 5 seconds left", "the traffic light ahead is green, you can pass", etc.

[0144] Step S304, obtaining a listening position designated by the user; or determining the listening position based on the position of the user in the vehicle.

[0145] This step can be performed after an obstacle is detected. Specifically, when it is determined that there are obstacles around the vehicle based on data collected by sensors installed on the vehicle, the listening position specified by the user is obtained, or the position of the user in the vehicle is obtained, and the listening position is determined based on the position of the user.

[0146] The head of the user sitting in the driver's or front passenger's seat may be selected as the listening position by default.

[0147] When there are multiple users in the vehicle, one user can be selected from the multiple users, such as the target user, and the head of the target user can be used as the listening position. Alternatively, the average or weighted average of the head coordinates of multiple users can be determined as the listening position. The weight during weighting can be determined based on the user priority and the position where the user is sitting. For example, the weight of the front seat is higher than that of the back seat, and the weight of the main driver's seat is higher than that of the co-driver's seat. The higher the user priority, the higher the weight of the user's head coordinates.

[0148] Optionally, the method further includes:

[0149] When a plurality of users are seated in the vehicle, the listening position is determined based on the seat position of each of the users, the position priority of the seat position of each of the users, and the priority of each of the users.

[0150] The weight of the user's seating position may be determined based on the position priority and the user's priority, and the listening position may be determined based on a weighted average of the seating positions of the users.

[0151] In some embodiments, the score of the seat of each user may be calculated based on the position priority and the user priority, and the listening position may be determined based on the seat of the user with the highest score.

[0152] Step S305: establishing a spherical coordinate system with the listening position as the origin, and determining a conversion relationship between the spherical coordinate system and the coordinate system of the vehicle.

[0153] Step S306: Based on the conversion relationship, coordinate conversion is performed on the coordinates of the obstacle, and the coordinates of the obstacle are converted into coordinates in a spherical coordinate system with the listening position as the origin, so as to obtain coordinates of multiple sampling points on the obstacle in the spherical coordinate system.

[0154] Assuming the conversion relationship is T, the coordinate r1 of the obstacle in the spherical coordinate system with the listening position as the origin and the coordinate r2 of the obstacle in the vehicle's coordinate system satisfy: r2 = T × r1.

[0155] Suppose the coordinates of a sampling point s on an obstacle are (x, y, z), and the coordinates of the listening position (such as the driver's head position) in the coordinate system are (x0, y0, z0). Establish a spherical coordinate system with the listening position as the origin, then the coordinates of the sampling point s on the obstacle in the spherical coordinate system are for:

[0156]

[0157]

[0158]

[0159] Step S307 , taking each of the sampling points as a point sound source, taking the coordinates of each of the sampling points in the spherical coordinate system as point sound source coordinates, and rendering the target prompt sound based on a preset rendering algorithm.

[0160] The sampling point is a point on the obstacle, and the coordinates of the obstacle in the spherical coordinate system include the coordinates of multiple sampling points on the obstacle in the spherical coordinate system.

[0161] The preset rendering algorithm may be a specified or default rendering algorithm, such as the HOA algorithm.

[0162] In some embodiments, in order to improve the rendering effect of the obstacle prompt sound, a rendering algorithm can be adapted for the obstacle. Specifically, the rendering algorithm of the target prompt sound corresponding to the obstacle can be adaptively determined based on the attribute information of the obstacle or the prompt scene.

[0163] Optionally, the method further includes:

[0164] Determine whether the moving speed of the obstacle is less than a preset speed; if so, determine that the preset rendering algorithm is a first rendering algorithm; if not, determine that the preset rendering algorithm is a second rendering algorithm; wherein the first rendering algorithm is a sound bed rendering algorithm or a panoramic sound rendering algorithm; and the second rendering algorithm is a panning algorithm or a wave field synthesis rendering algorithm.

[0165] The rendering algorithm can be First Order Ambisonics (FOA) or High Order Ambisonics (HOA). HOA uses high-order spherical harmonic functions, such as second-order and third-order, to improve the spatial resolution of the rendering algorithm, enrich the details of the sound and image, and improve the stereo effect of the prompt sound.

[0166] Panning algorithms are rendering algorithms that use panning, such as the basic panning algorithm, which performs sound rendering through simple panning, or improved panning algorithms, such as the Vector Based Amplitude Panning (VBAP) and the Distance Based Amplitude Panning (DBAP).

[0167] Wave Field Synthesis (WFS) is a technology that uses a speaker array to reconstruct the spatial sound field and reproduce the real sound field at a specified listening position.

[0168] By differentiating the rendering algorithm for different obstacles, the three-dimensional rendering effect of the obstacle warning sound is improved, and a three-dimensional sound field can be realized in the car.

[0169] Figure 5 A schematic diagram of a target prompt sound rendering effect provided in an embodiment of the present application, such as Figure 5 As shown, there is an obstacle in front of the right side of the vehicle ( Figure 5 The obstacle is represented by a solid circle in the figure), such as a pedestrian crossing the road. Based on the distance between the obstacle and the vehicle, the direction relative to the vehicle, and the moving speed of the obstacle, the prompt content of the target prompt sound is determined to be "Beware of obstacles". By taking the obstacle as the sound source, rendering the target prompt sound, the sound of "Beware of obstacles" emitted by the speaker array in the vehicle reaching the listening position simulates the sound of "Beware of obstacles" emitted by the sound source at the obstacle to reach the listening position, so that the user at the listening position, such as the driver, can judge the location of the obstacle after hearing the prompt sound of "Beware of obstacles". Figure 5 The dotted circle in the figure indicates the position of the obstacle determined by the user.

[0170] In this embodiment, when an obstacle is detected, the listening position is obtained and a spherical coordinate system with the listening position as the origin is established. At the same time, based on the data of the obstacle collected by the sensor, the coordinates of the obstacle in the vehicle coordinate system are obtained, and the coordinates of each point on the obstacle are converted into spherical coordinates through coordinate conversion to facilitate prompt sound rendering; when matching the prompt sound for the obstacle, the properties of the obstacle itself are fully considered, and the prompt scene corresponding to the obstacle is determined. Based on the prompt scene, a matching prompt sound is searched from the prompt sound library as the prompt sound of the obstacle, thereby improving the accuracy of the prompt sound and matching the prompt sound with the scene corresponding to the obstacle; when rendering the prompt sound corresponding to the obstacle, multiple point sound sources are used to describe the sound source corresponding to the obstacle, and based on a preset rendering algorithm, the effect of the target prompt sound emitted by the sound source reaching the listening position is rendered, so that the user at the listening position can identify the approximate direction of the obstacle based on the target prompt sound heard, thereby quickly determining the position of the obstacle, avoiding collision between the vehicle and the obstacle, and improving driving safety.

[0171] Figure 6 A flowchart of another obstacle prompting method provided in an embodiment of the present application is provided in this embodiment. Figure 2 Based on the embodiment shown, step S202 and step S203 are further refined as follows: Figure 6 As shown, the obstacle prompting method provided in this embodiment specifically includes the following steps:

[0172] Step S601, based on the data collected by the vehicle sensor, determine the target prompt sound corresponding to the obstacle from the prompt sound library, and determine the coordinates of the obstacle in the coordinate system of the vehicle.

[0173] Step S602: establishing an obstacle model of the obstacle based on the coordinates of a plurality of sampling points on the obstacle in the coordinate system of the vehicle.

[0174] The obstacle model may be a model corresponding to the geometric shape of the obstacle. The obstacle model is a virtual model corresponding to the obstacle, which is used to represent the geometric shape of the obstacle and may also be used to represent the attribute information of the obstacle.

[0175] Based on the coordinates of multiple sampling points on the outer surface of the obstacle, the outer contour of the obstacle can be obtained by aggregation, and an obstacle model of the obstacle can be established.

[0176] In some embodiments, the obstacle model of the obstacle may also be established in combination with the attribute information of the obstacle, such as type, moving speed, temperature, etc.

[0177] Step S603: performing coordinate transformation on the obstacle model to obtain the obstacle model in a spherical coordinate system with the listening position as the origin.

[0178] Step S604, determine whether the moving speed of the obstacle is less than a preset speed; if so, execute step S605; if not, jump to step S606.

[0179] The preset speed is a smaller speed, such as 5 km / h, 10 km / h or other values.

[0180] The moving speed relied upon for determining the rendering algorithm to be adopted may be the moving speed of the obstacle relative to the vehicle, or the moving speed of the obstacle relative to the ground.

[0181] Step S605 , using the obstacle model after coordinate transformation as a sound source model, and rendering the target prompt sound based on a first rendering algorithm.

[0182] Step S606: Using the obstacle model after coordinate transformation as a sound source model, and rendering the target prompt sound based on a second rendering algorithm.

[0183] Among them, the first rendering algorithm is a sound bed rendering algorithm or a panoramic sound rendering algorithm; the second rendering algorithm is a panning algorithm or a wave field synthesis rendering algorithm.

[0184] In this embodiment, an obstacle model is first established based on the coordinates of multiple points on the obstacle. The method of performing coordinate transformation on the obstacle model is more efficient than the method of performing coordinate transformation point by point. The obstacle model is used as the sound source model for rendering, which avoids rendering of multiple point sound sources and improves rendering efficiency.

[0185] The embodiment of the present application also provides an obstacle prompting device, comprising:

[0186] A prompt sound determination module is used to determine the target prompt sound corresponding to the obstacle from the prompt sound library based on the data collected by the vehicle sensor; a coordinate determination module is used to determine the coordinates of the obstacle in the coordinate system of the vehicle based on the data collected by the vehicle sensor; a coordinate transformation module is used to perform coordinate transformation on the obstacle, and transform the coordinates of the obstacle into coordinates in a spherical coordinate system with the listening position as the origin; a prompt sound rendering module is used to render the target prompt sound with the coordinates of the obstacle in the spherical coordinate system as the sound source coordinates.

[0187] In some embodiments, the prompt tone determination module includes:

[0188] An attribute information determination unit is used to determine the attribute information of the obstacle based on data collected by the vehicle sensor, wherein the attribute information is used to describe at least one of the type of the obstacle, the moving speed, and the distance from the vehicle; a prompt scene determination unit is used to determine the prompt scene corresponding to the obstacle based on the attribute information of the obstacle; and a prompt sound determination unit is used to determine, from a prompt sound library, the prompt sound corresponding to the prompt scene as the target prompt sound corresponding to the obstacle.

[0189] In some embodiments, the prompting scene determination unit is specifically configured to:

[0190] If the type of the obstacle is a pedestrian and the distance between the obstacle and the vehicle is greater than the first distance, the prompt sound scene corresponding to the obstacle is determined to be a far-field pedestrian scene; if the type of the obstacle is a pedestrian and the distance between the obstacle and the vehicle is less than or equal to the first distance, the prompt scene corresponding to the obstacle is a near-field pedestrian scene.

[0191] In some embodiments, the prompting scene determination unit is specifically configured to:

[0192] If the distance between the obstacle and the vehicle is greater than or equal to a preset distance, or if the distance between the obstacle and the vehicle is less than the preset distance, the moving speed of the obstacle relative to the vehicle is less than or equal to the set speed, and the distance between the obstacle and the vehicle is less than or equal to the safe braking distance of the obstacle relative to the moving speed of the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a stationary prompt scene; if the distance between the obstacle and the vehicle is less than the preset distance, and the distance between the obstacle and the vehicle is greater than the safe braking distance of the obstacle relative to the moving speed of the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a dangerous moving prompt scene; if the distance between the obstacle and the vehicle is less than the preset distance, the moving speed of the obstacle relative to the vehicle is greater than the set speed, and the distance between the obstacle and the vehicle is less than or equal to the safe braking distance of the obstacle relative to the moving speed of the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a high-speed moving prompt scene.

[0193] In some embodiments, the coordinates of the obstacle in the spherical coordinate system include the coordinates of multiple sampling points on the obstacle in the spherical coordinate system, and the prompt sound rendering module is specifically used to:

[0194] The target prompt sound is rendered based on a preset rendering algorithm, taking each of the sampling points as a point sound source and taking the coordinates of each of the sampling points in the spherical coordinate system as point sound source coordinates.

[0195] In some embodiments, the coordinate transformation module is specifically used to:

[0196] An obstacle model of the obstacle is established based on the coordinates of multiple sampling points on the obstacle in the coordinate system of the vehicle; and the obstacle model is subjected to coordinate transformation to obtain the obstacle model in a spherical coordinate system with the listening position as the origin.

[0197] Correspondingly, the prompt sound rendering module is specifically used for:

[0198] The obstacle model after coordinate transformation is used as the sound source model, and the target prompt sound is rendered based on a preset rendering algorithm.

[0199] In some embodiments, the device further comprises a preset rendering algorithm determination module for

[0200] Determine whether the moving speed of the obstacle is less than a preset speed; if so, determine that the preset rendering algorithm is a first rendering algorithm; if not, determine that the preset rendering algorithm is a second rendering algorithm; wherein the first rendering algorithm is a sound bed rendering algorithm or a panoramic sound rendering algorithm; and the second rendering algorithm is a panning algorithm or a wave field synthesis rendering algorithm.

[0201] In some embodiments, the device further comprises a listening position acquisition module, which is used to:

[0202] The listening position specified by the user is obtained; or the listening position is determined based on the position of the user sitting in the vehicle.

[0203] In some embodiments, the apparatus further comprises a listening position determination module, configured to:

[0204] When a plurality of users are seated in the vehicle, the listening position is determined based on the seat position of each of the users, the position priority of the seat position of each of the users, and the priority of each of the users.

[0205] The above device embodiment is an embodiment corresponding to the above method embodiment, and has the same technical effect as the method embodiment. The detailed description of the device embodiment can refer to the detailed description of the above method embodiment, and will not be repeated here.

[0206] Figure 7 The structure diagram of a vehicle audio system provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the vehicle audio system includes: a sensor and a renderer. The sensors are usually multiple and are used to collect data of obstacles outside the vehicle; the renderer is used to execute the obstacle prompt method provided in any embodiment of the present application to achieve the rendering of the target prompt sound corresponding to the obstacle.

[0207] A prompt sound library is stored in the vehicle audio system, and the renderer selects the target prompt sound corresponding to the obstacle from the prompt sound library and renders it.

[0208] In some embodiments, the vehicle audio system further includes a speaker array on the vehicle to play the rendered target prompt sound through the speaker array.

[0209] In some embodiments, the vehicle audio system includes an ADAS, which is used to determine whether there is an obstacle based on data collected by a sensor. If so, the data corresponding to the obstacle collected by the sensor is sent to a renderer to generate a warning sound for the obstacle.

[0210] Figure 8 The present invention provides a schematic diagram of an obstacle warning system provided in an embodiment of the present invention. The obstacle warning system is arranged in a vehicle and is used for obstacle detection, prompt tone matching and prompt tone playing. Figure 8 As shown, the obstacle warning system includes a sensor system, an obstacle attribute recognition module, a prompt sound matching module, a rendering module and a playback system.

[0211] The sensor system includes multiple sensors installed on the vehicle, such as infrared sensors, radar sensors, image sensors, ultrasonic sensors, microphones, etc., to collect data on obstacles, including but not limited to raw data corresponding to attributes such as size, position, moving speed, temperature, audio data, etc.

[0212] The obstacle attribute recognition module is used to determine the attribute information of the obstacle, such as the category, material, size, orientation, speed, audio data, etc. of the obstacle based on the original data of the obstacle fed back by the sensor system, such as signal strength, image, distance, temperature, size, etc.

[0213] The prompt sound matching module is used to determine the prompt sound that matches the attribute information of the obstacle from the prompt sound library as the prompt sound corresponding to the obstacle, that is, the target prompt sound.

[0214] The rendering module is responsible for converting the coordinates of the obstacle into spherical coordinates in a spherical coordinate system with the listening position as the origin, and using the spherical coordinates of the obstacle as the sound source coordinates, rendering the target prompt sound, and obtaining the audio signals corresponding to each speaker in the playback system.

[0215] The playback system includes a speaker array in the vehicle, such as a 7.1-channel speaker array, through which the target prompt sound is played.

[0216] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the processor implements a method provided in any embodiment of the present application.

[0217] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the method provided in any embodiment of the present application.

[0218] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0219] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An obstacle prompting method, characterized in that: include: Based on the data collected by the vehicle sensor, determine the target prompt sound corresponding to the obstacle from the prompt sound library, and determine the coordinates of the obstacle in the coordinate system of the vehicle; Performing coordinate transformation on the obstacle, transforming the coordinates of the obstacle into coordinates in a spherical coordinate system with the listening position as the origin; The target prompt sound is rendered using the coordinates of the obstacle in the spherical coordinate system as the sound source coordinates.

2. The method according to claim 1, characterized in that Based on the data collected by the vehicle sensor, determining the target prompt sound corresponding to the obstacle from the prompt sound library includes: Determine attribute information of the obstacle based on data collected by the vehicle sensor, where the attribute information is used to describe at least one of the type of the obstacle, the moving speed, and the distance from the vehicle; Based on the attribute information of the obstacle, determining a prompt scene corresponding to the obstacle; The prompt sound corresponding to the prompt scene is determined from the prompt sound library as the target prompt sound corresponding to the obstacle.

3. The method according to claim 2, characterized in that Determining a prompt scene corresponding to the obstacle based on the attribute information of the obstacle includes: If the type of the obstacle is a pedestrian and the distance between the obstacle and the vehicle is greater than a first distance, determining that the prompt sound scene corresponding to the obstacle is a far-field pedestrian scene; If the type of the obstacle is a pedestrian and the distance between the obstacle and the vehicle is less than or equal to the first distance, the prompt scene corresponding to the obstacle is a near-field pedestrian scene.

4. The method according to claim 2, characterized in that: Determining a prompt scene corresponding to the obstacle based on the attribute information of the obstacle includes: If the distance between the obstacle and the vehicle is greater than or equal to a preset distance, or if the distance between the obstacle and the vehicle is less than the preset distance, the moving speed of the obstacle relative to the vehicle is less than or equal to a set speed, and the distance between the obstacle and the vehicle is less than or equal to a safe braking distance at the moving speed of the obstacle relative to the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a stationary prompt scene; If the distance between the obstacle and the vehicle is less than a preset distance, and the distance between the obstacle and the vehicle is greater than a safe braking distance of the obstacle relative to the moving speed of the vehicle, determining that the warning sound scene corresponding to the obstacle is a dangerous movement warning scene; If the distance between the obstacle and the vehicle is less than a preset distance, the moving speed of the obstacle relative to the vehicle is greater than a set speed, and the distance between the obstacle and the vehicle is less than or equal to the safe braking distance of the obstacle at the moving speed of the vehicle, then the prompt sound scene corresponding to the obstacle is determined to be a high-speed moving prompt scene.

5. The method according to claim 1, characterized in that: The coordinates of the obstacle in the spherical coordinate system include the coordinates of a plurality of sampling points on the obstacle in the spherical coordinate system, and the coordinates of the obstacle in the spherical coordinate system are used as the sound source coordinates, and rendering the target prompt sound includes: The target prompt sound is rendered based on a preset rendering algorithm, taking each of the sampling points as a point sound source and taking the coordinates of each of the sampling points in the spherical coordinate system as point sound source coordinates.

6. The method according to claim 1, characterized in that Performing coordinate transformation on the obstacle to transform the coordinates of the obstacle into coordinates in a spherical coordinate system with the listening position as the origin includes: Establishing an obstacle model of the obstacle based on coordinates of a plurality of sampling points on the obstacle in a coordinate system of the vehicle; Performing coordinate transformation on the obstacle model to obtain the obstacle model in a spherical coordinate system with the listening position as the origin; The target prompt sound is rendered using the coordinates of the obstacle in the spherical coordinate system as the sound source coordinates, including: The obstacle model after coordinate transformation is used as the sound source model, and the target prompt sound is rendered based on a preset rendering algorithm.

7. The method according to claim 5 or 6, characterized in that: The method further comprises: Determining whether the moving speed of the obstacle is less than a preset speed; If yes, determining that the preset rendering algorithm is the first rendering algorithm; If not, determining that the preset rendering algorithm is the second rendering algorithm; Among them, the first rendering algorithm is a sound bed rendering algorithm or a panoramic sound rendering algorithm; the second rendering algorithm is a panning algorithm or a wave field synthesis rendering algorithm.

8. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Get the listening position specified by the user; or, The listening position is determined based on a position of a user sitting in the vehicle.

9. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: When a plurality of users are seated in the vehicle, the listening position is determined based on the seat position of each of the users, the position priority of the seat position of each of the users, and the priority of each of the users.

10. A vehicle audio system, arranged in a vehicle, characterized in that: include: Sensors and renderers; The sensor is used to collect data of obstacles outside the vehicle; The renderer is used to execute the method provided in any one of claims 1 to 9 to achieve rendering of the target prompt sound corresponding to the obstacle.

Images