Sound source positioning method and device, vehicle and storage medium
By installing a microphone array on the vehicle, multiple candidate directions of the sound source are determined and signal delay is compensated, which solves the problem of inaccurate sound source direction identification in traditional traffic modes, achieves more efficient sound source localization and avoidance, and improves driving safety.
Patent Information
- Application Number
- CN202511518306.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-22
- Publication Date
- 2026-02-17
AI Technical Summary
In traditional traffic patterns, vehicles often fail to accurately identify the direction of emergency vehicle sound sources, impacting road traffic efficiency and public safety.
By determining multiple candidate directions of the sound source, signal delay compensation is performed using a microphone array to enhance signal strength, and the direction of the sound source is determined based on the signal strength.
It enables all-around identification of sound source direction, improves the comprehensiveness and accuracy of sound source localization, enhances anti-interference ability, helps drivers or autonomous driving systems to make quick and accurate avoidance, and improves driving safety.
Smart Images
Figure CN121541135A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the fields of signal recognition, signal positioning, and vehicle technology, and in particular to a sound source positioning method, device, vehicle, and storage medium. Background Technology
[0002] With the significant increase in urban road traffic density, interactions between vehicles and emergency vehicles (such as ambulances, fire trucks, and police cars) are becoming more frequent, becoming a key factor affecting road traffic efficiency and public safety. In traditional traffic patterns, vehicles mainly rely on drivers to manually avoid emergency vehicles by hearing their sounds. Summary of the Invention
[0003] This disclosure provides a sound source localization method, apparatus, vehicle, and computer-readable storage medium to at least solve the problem of fixed sound source direction identification in related technologies. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, a sound source localization method is provided, comprising: determining a plurality of preset candidate directions of a sound source, wherein the candidate directions include a plurality of potential directions of the sound source; performing time delay compensation on signals collected by microphones according to first projection positions of different microphones on the candidate directions to obtain an enhanced signal corresponding to the candidate directions; and determining the sound source direction based on the signal strength of the enhanced signal corresponding to the candidate directions.
[0004] According to a second aspect of the present disclosure, a sound source localization device is provided, comprising: a determining module, configured to determine a plurality of preset candidate directions of a sound source, wherein the candidate directions include a plurality of potential directions of the sound source; a compensation module, configured to perform time delay compensation on signals collected by microphones according to a first projection position of different microphones on the candidate directions to obtain an enhanced signal corresponding to the candidate directions; and a localization module, configured to determine the direction of the sound source based on the signal strength of the enhanced signal corresponding to the candidate directions.
[0005] According to a third aspect of the present disclosure, a vehicle is provided, including a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the steps of the method described in the first aspect of the present disclosure.
[0006] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the steps of the method described in the first aspect of the present disclosure.
[0007] The technical solution provided by the embodiments of this disclosure brings at least the following beneficial effects: By determining multiple preset candidate directions of the sound source, and based on the first projection position of different microphones in the candidate directions, the signal delay of the microphone is determined according to the first projection position, and the signal collected by the microphone is time-delay compensated according to the signal delay to obtain the enhanced signal corresponding to the candidate direction. Furthermore, based on the signal strength of the enhanced signal corresponding to the candidate direction, the sound source direction can be determined. Therefore, this solution can achieve omnidirectional sound source direction identification, improving the comprehensiveness of sound source localization. By enhancing the signal collected by the microphone, the signal strength can be increased, improving the anti-interference capability of sound source localization based on signal strength, thereby improving the accuracy of sound source localization. Accurately identifying the sound source direction can help drivers or autonomous driving systems to avoid obstacles more quickly and accurately, improving driving safety.
[0008] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0009] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.
[0010] Figure 1 This is a flowchart illustrating a sound source localization method according to an exemplary embodiment.
[0011] Figure 2 This is a flowchart illustrating a sound source localization method according to another exemplary embodiment.
[0012] Figure 3 This is a flowchart illustrating the localization of a sound source according to an exemplary embodiment.
[0013] Figure 4 This is a flowchart illustrating a sound source localization method according to another exemplary embodiment.
[0014] Figure 5 This is a flowchart illustrating a sound source localization method according to another exemplary embodiment.
[0015] Figure 6 This is a block diagram illustrating a sound source localization device according to an exemplary embodiment.
[0016] Figure 7 This is a block diagram illustrating a vehicle according to an exemplary embodiment. Detailed Implementation
[0017] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0018] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0019] Figure 1 This is a flowchart illustrating a sound source localization method according to an exemplary embodiment, such as... Figure 1 As shown, the sound source localization method of this disclosure includes the following steps: S101, determine multiple candidate directions preset for the sound source, wherein the candidate directions include multiple directions used to represent the potential directions of the sound source.
[0020] It should be noted that the execution subject of the sound source localization method in this embodiment is an electronic device, which can be a vehicle-side device or an in-vehicle device, such as an electronic device equipped with a vehicle control system. The sound source localization method in this embodiment can be executed by the sound source localization device in this embodiment, which can be configured in any electronic device to execute the sound source localization method in this embodiment.
[0021] It should be noted that the sound source in this embodiment refers to a sound source in the external environment of the vehicle. The sound source includes, but is not limited to, natural sound sources, such as thunder and so on, traffic-related sound sources, such as emergency vehicle sounds, rail transit sounds, and environmental sound sources such as industrial facility sounds.
[0022] In some embodiments, multiple candidate directions can be pre-set based on the vehicle according to different directions and angles, wherein the candidate directions are used to indicate the possible directions from which the sound source may come. The candidate directions can cover a 360-degree space including the front, rear, sides, and diagonal directions of the vehicle.
[0023] In some embodiments, when presetting candidate directions, the density of the candidate directions can also be determined so that multiple candidate directions can be preset according to different densities. A higher density indicates a larger number of candidate directions, and a lower density indicates a smaller number of candidate directions.
[0024] S102, based on the first projection position of different microphones in the candidate direction, perform time delay compensation on the signal collected by the microphone to obtain the enhanced signal corresponding to the candidate direction.
[0025] In some embodiments, multiple microphones can be installed at different locations on the vehicle to form a microphone array, and the microphones can be used to collect sound signals. In this embodiment of the disclosure, the vehicle's microphones can collect sound signals emitted by a sound source.
[0026] In some embodiments, any candidate direction is determined from a plurality of preset candidate directions, and a microphone is projected onto the candidate direction to determine a first projection position of the microphone in the candidate direction. The first projection position indicates the projection distance between the microphone and the candidate direction.
[0027] In some embodiments, the coordinate information of the microphone in the candidate direction can be determined based on the microphone's coordinate information and the azimuth angle corresponding to the candidate direction, and the coordinate information can be used as the first projection position of the microphone in the candidate direction.
[0028] For example, such as Figure 3 As shown, Figure 3 The microphones include microphone A and microphone B, and the azimuth angle corresponding to the candidate direction is... θ Then, based on the coordinates of microphone A ( a x ,a y ) and azimuth θ Determine the first projection position of microphone A in the candidate direction. And according to the coordinates of microphone B ( b x ,b y ) and azimuth θ Determine the first projection position of microphone B in the candidate direction. .
[0029] In some embodiments, after determining the first projection position of the microphone in the candidate direction, the signal delay corresponding to the microphone can be determined based on the first projection position, so as to perform delay compensation on the signal collected by the microphone based on the signal delay.
[0030] In some embodiments, to quantify the time delay characteristics of a sound source on a microphone array, the time difference between the sound signals can be calculated using the geometric projection of the microphones in candidate directions and the speed of sound in air, and this time difference can be used as the signal delay for each microphone. For example, if microphone A and microphone B exist, the time difference from when the signal arrives at microphone A to when it arrives at microphone B can be calculated as the signal delay.
[0031] In some embodiments, a reference microphone can be determined from among different microphones, and the signal delay can be determined based on the second projection position of the reference microphone in the candidate direction and the first projection position. Determining the reference microphone to calculate the signal delay can improve the accuracy of the delay calculation.
[0032] In some embodiments, the projection length of the microphone in the candidate direction can be determined based on the second projection position and the first projection position, and the signal delay can be calculated based on the projection length and the speed of sound. Continuing with... Figure 3 For example, if microphone A is the reference microphone, the second projection position is... Then we can calculate and Projection length between Therefore, based on and speed of sound c Calculate the signal delay of microphone B. If the vehicle also includes microphone C, its first projected position in the candidate direction is... Then you can calculate and Projection length between Therefore, based on and speed of sound c Calculate the signal delay of microphone C.
[0033] In some embodiments, the first projection position is used to indicate the projection distance between the microphone and the candidate direction. A reference microphone can then be determined from different microphones based on the first projection positions corresponding to different microphones. By determining the projection distances of different microphones in the candidate direction, and selecting the first projection position corresponding to the largest projection distance from the projection distances, the microphone corresponding to this first projection position is used as the reference microphone.
[0034] In some embodiments, after determining the signal delay, delay compensation can be performed on the signal acquired by the microphone to obtain the enhanced signal corresponding to the candidate direction. Optionally, the acquired signal can be compensated based on the signal delay corresponding to the microphone to obtain a compensated signal, and the compensated signals corresponding to different microphones can be fused to obtain the enhanced signal in the candidate direction.
[0035] For example, if a vehicle has two microphones: microphone A and microphone B, with microphone A as the reference microphone, then the compensation signal corresponding to microphone B is the enhanced signal. If the vehicle has multiple microphones: microphone A, microphone B, and microphone C, with microphone A as the reference microphone, then the compensation signals corresponding to microphone B and microphone C can be fused to obtain the enhanced signal in the candidate direction.
[0036] S103, determine the sound source direction based on the signal intensity of the enhanced signal corresponding to the candidate direction.
[0037] In some embodiments, step S101 may be performed for each candidate direction to determine the enhancement signal corresponding to each candidate direction, and the signal strength of the enhancement signal corresponding to each candidate direction may be determined based on the enhancement signal.
[0038] In some embodiments, the signal strengths of the enhanced signals corresponding to the candidate directions can be compared to determine the target enhanced signal with the largest signal strength from the signal strengths, and the candidate direction corresponding to the target enhanced signal can be taken as the sound source direction.
[0039] In some embodiments, the signal amplitude and signal energy of the enhanced signal can be used as the signal strength of the enhanced signal. That is, the target enhanced signal with the largest signal amplitude can be selected, and the candidate direction corresponding to the target enhanced signal can be used as the sound source direction. Alternatively, the target enhanced signal with the largest signal energy can be selected, and the candidate direction corresponding to the target enhanced signal can be used as the sound source direction.
[0040] Exemplarily, the sound source localization method of this embodiment is explained using the sound of an emergency vehicle as the sound source. By determining any one of the preset candidate directions 1, 2, 3, and 4 as candidate direction 1, and obtaining the first projection position of different microphones on candidate direction 1, the signal delay corresponding to the microphone is calculated based on the first projection position. This allows for delay compensation of the signal collected by the microphone, resulting in an enhanced signal 1 corresponding to candidate direction 1. By determining the enhanced signals 1, 2, 3, and 4 corresponding to each candidate direction, and determining the signal strengths 1, 2, 3, and 4 of the enhanced signals corresponding to the candidate directions, if signal strength 1 is the largest, then candidate direction 1 corresponding to enhanced signal 1 is the direction of the emergency vehicle sound.
[0041] The sound source localization method provided in this disclosure determines multiple preset candidate directions of the sound source, and determines the signal delay of the microphone based on the first projection position of different microphones in the candidate directions. The signal delay is then compensated for by the microphones to obtain an enhanced signal corresponding to the candidate directions. Further, the sound source direction can be determined based on the signal strength of the enhanced signal corresponding to the candidate directions. Therefore, this solution can achieve omnidirectional sound source direction identification, improving the comprehensiveness of sound source localization. By enhancing the signal collected by the microphones, the signal strength is increased, improving the anti-interference capability of sound source localization based on signal strength, thereby improving the accuracy of sound source localization. Accurate sound source direction identification can help drivers or autonomous driving systems to avoid obstacles more quickly and accurately, improving driving safety.
[0042] Figure 2 This is a flowchart illustrating a sound source localization method according to an exemplary embodiment, such as... Figure 2 As shown, the sound source localization method of this disclosure includes the following steps: S201, determine multiple candidate directions preset for the sound source, wherein the candidate directions include multiple directions used to represent the potential directions of the sound source.
[0043] S202, determine the reference position corresponding to the candidate direction.
[0044] In some embodiments, to ensure consistency in the process of calculating the signal delay corresponding to the microphone, a reference position corresponding to the candidate direction can be determined, and the signal delay corresponding to the microphone can be calculated based on the reference position and the first projection position of the microphone in the candidate direction.
[0045] In some embodiments, a reference microphone can be determined from different microphones, and a reference position can be determined based on the reference microphone. Alternatively, the reference microphone can be determined based on the projected position of the microphone in the candidate direction.
[0046] In other words, by projecting different microphones onto the candidate direction, the projection position of different microphones in the candidate direction can be determined, and based on the projection position of different microphones in the candidate direction, the reference microphone corresponding to the candidate direction can be determined from the different microphones, where the location of the reference microphone is the reference position corresponding to the candidate direction.
[0047] For example, with Figure 3 For example, if the reference microphone corresponding to the candidate direction is microphone A, then microphone A is used as the position ( a x ,a y ) is the reference position corresponding to the candidate direction. If the reference microphone corresponding to the candidate direction is microphone B, then microphone B is used as the reference position at ( b x ,b y () represents the reference position corresponding to the candidate direction.
[0048] In some embodiments, the projection position can be used to indicate the projection distance of the microphone in the candidate direction, so that the reference microphone corresponding to the candidate direction can be determined from different microphones based on the projection distance. Optionally, the projection distance of different microphones in the candidate direction can be determined based on the projection positions of different microphones in the candidate direction. Figure 3 As shown, taking microphone A as an example, the projected position of microphone A in the candidate direction is... Then A and The distance between them is the projection distance of microphone A in the candidate direction.
[0049] In some embodiments, the projection distance of the microphone in the candidate direction can be calculated based on the microphone's coordinate information and the coordinate information of the microphone's projection position in the candidate direction, thereby determining the reference microphone corresponding to the candidate direction based on the projection distance of different microphones in the candidate direction.
[0050] In some embodiments, the projection distances can be sorted according to their magnitude, and the reference microphone corresponding to the candidate direction can be determined based on the sorting result. For example, the projection distances can be sorted in descending order to determine the microphone with the largest projection distance from the sorting result as the reference microphone corresponding to the candidate direction.
[0051] For example, the projection distances can be sorted in ascending order to determine the microphone with the smallest projection distance from the sorting results, which can then be used as the reference microphone for the candidate direction.
[0052] For example, a target projection distance can be preset, and the microphone whose projection distance in the candidate direction is the target projection distance can be used as the reference microphone for the candidate direction. Optionally, if no microphone has a target projection distance in its projection distance in the candidate direction, the difference between the projection distance and the target projection distance can be obtained to determine the projection distance corresponding to the minimum difference, and the microphone corresponding to this projection distance can be used as the reference microphone for the candidate direction.
[0053] S203, determine the signal delay corresponding to the microphone based on the first projection position and the reference position of the microphone in the candidate direction.
[0054] It should be noted that the first projected position of the microphone in the candidate direction can be calculated based on the microphone's coordinate information. This coordinate information is based on the Cartesian coordinate system of the vehicle's center, such as... Figure 3 As shown, with the center of the vehicle as the origin, the positive x-axis is to the right and the positive y-axis is above.
[0055] Since there is an angle between the candidate direction and the x-axis, which is the azimuth angle corresponding to the candidate direction, the first projection position and the reference position of the microphone in the candidate direction can also be calculated based on the azimuth angle corresponding to the candidate direction.
[0056] In other words, by determining the microphone's coordinate information and based on the azimuth and coordinate information corresponding to the candidate direction, the first projection position of the microphone in the candidate direction is determined.
[0057] In some embodiments, when sound propagates through the air, the time it takes for microphones at different locations to receive the signal is different. The signal delay can be determined by determining the distance between the first projection position and the reference position, and based on that distance. Optionally, since the reference position refers to the position of a reference microphone, the signal delay corresponding to the microphone can be determined by determining a second projection position of the reference position in a candidate direction, based on the first projection position and the second projection position.
[0058] In some embodiments, a reference microphone corresponding to a reference position is determined, and the first projection position of the reference microphone in the candidate direction is used as the second projection position of the reference position in the candidate direction.
[0059] Furthermore, the propagation distance difference between the microphone and the reference position can be determined based on the first projection position and the second projection position, and the signal delay corresponding to the microphone can be determined based on the propagation distance difference and the signal propagation speed.
[0060] For example, Figure 3 In the diagram, microphone A is the reference microphone, then the second projection position is... By obtaining the first projection position of microphone B With the second projection position Difference in propagation distance between Furthermore, based on the difference in propagation distance and signal propagation speed c Determine signal delay .
[0061] S204, based on the signal delay corresponding to the microphone, performs delay compensation on the signal collected by the microphone to obtain the enhanced signal corresponding to the candidate direction.
[0062] In some embodiments, a compensation signal corresponding to a microphone can be determined based on the signal delay of the microphone, and an enhanced signal corresponding to the candidate direction can be obtained based on the compensation signals of different microphones in the candidate direction. For example, the compensation signals can be fused to obtain the enhanced signal.
[0063] Optionally, the signal acquired by the microphone can be time-delay compensated based on the signal delay corresponding to the microphone to obtain the compensated signal of the microphone, and then the compensated signal of the microphone can be fused to obtain the enhanced signal corresponding to the candidate direction.
[0064] In some embodiments, the compensation signals can be added together to obtain an intermediate result, and the intermediate result can be averaged to achieve the fusion of the compensation signals and obtain an enhanced signal.
[0065] In some embodiments, if there are two microphones on the vehicle, one of which is a reference microphone, the enhanced signal corresponding to the candidate direction is the compensation signal corresponding to the other microphone. If there are more than two microphones on the vehicle, the enhanced signal corresponding to the candidate direction is a fused signal of the compensation signals corresponding to the microphones other than the reference microphone.
[0066] For example, if there are microphones A, B, and C, with microphone A as the reference microphone, then the compensation signal 1 corresponding to microphone B and the compensation signal 2 corresponding to microphone C can be fused to obtain the enhanced signal in the candidate direction.
[0067] In some embodiments, signal compensation can be achieved by adding the signal delay to the signal acquired by the microphone. Alternatively, a delay compensation filter can be used to compensate the signal acquired by the microphone to obtain a compensated signal.
[0068] In some embodiments, a delay compensation filter for the microphone can be determined in advance based on the signal delay corresponding to the microphone, and then the signal collected by the microphone can be compensated for the delay by the delay compensation filter to obtain the compensated signal corresponding to the microphone.
[0069] In some embodiments, a correspondence between signal delay and delay compensation filter can be established in advance based on different signal delays and delay compensation filters. By querying this correspondence, the signal delay corresponding to the microphone can be determined, and the delay compensation filter corresponding to the microphone can be identified.
[0070] S205, determine the sound source direction based on the signal intensity of the enhanced signal corresponding to the candidate direction.
[0071] The details of step S205 can be found in the above embodiments and will not be repeated here.
[0072] The sound source localization method provided in this disclosure determines the microphone signal delay based on a reference position in a candidate direction and the microphone's first projection position and the reference position. It then compensates for the signal delay in the microphone-acquired signal to obtain an enhanced signal corresponding to the candidate direction. By enhancing the microphone-acquired signal, the signal strength is increased, improving the anti-interference capability of sound source localization based on signal strength, thereby improving the accuracy of sound source localization.
[0073] Figure 3 This is a schematic diagram for locating the direction of a sound source. Figure 3 The vehicle has microphones A and B. The first projection position of microphone A in the candidate direction is determined. and the first projection position of microphone B in the candidate direction. This allows us to determine the projected distance of microphone A in the candidate direction. and the projected distance of microphone A in the candidate direction. Further based on and The size is selected from those with the largest projection distance. The corresponding microphone A is used as the reference microphone.
[0074] By determining the propagation distance difference between microphone B and microphone A Furthermore, based on the difference in propagation distance and signal propagation speed c Determine signal delay t θ Based on signal delay t θ A time delay compensation filter is determined, and the signal acquired by microphone B is compensated to obtain the compensated signal. x B This allows the compensation signal to be used. x B As an enhancement signal in the candidate direction x B .
[0075] Furthermore, the enhancement signal corresponding to each candidate direction is obtained, and the target enhancement signal with the largest signal intensity is determined based on the signal intensity of the enhancement signal. The candidate direction corresponding to the target enhancement signal is then taken as the sound source direction.
[0076] Figure 4 This is a flowchart illustrating a sound source localization method according to an exemplary embodiment, such as... Figure 4 As shown, the sound source localization method of this disclosure includes the following steps: S401, determine multiple candidate directions preset for the sound source, wherein the candidate directions include multiple directions used to represent the potential directions of the sound source.
[0077] S402, based on the first projection position of different microphones in the candidate direction, performs time delay compensation on the signal collected by the microphone to obtain the enhanced signal corresponding to the candidate direction.
[0078] The details of steps S401-S402 can be found in the above embodiments and will not be repeated here.
[0079] S403, extract the signal strength of the enhanced signal corresponding to the candidate direction, wherein the signal strength includes at least one of signal amplitude and signal energy.
[0080] In some embodiments, after obtaining the enhanced signal corresponding to the candidate direction, the sound source direction can be determined from a preset plurality of candidate directions based on the magnitude of the enhanced signal. Optionally, the magnitude of the enhanced signal can be determined based on the signal strength of the enhanced signal.
[0081] In some embodiments, signal strength may include at least one of signal amplitude and signal energy, wherein the signal amplitude reflects the instantaneous strength or maximum deviation of the signal, and the signal energy represents the total strength of the signal in time or space.
[0082] In some embodiments, the maximum value of the enhanced signal can be extracted as the signal amplitude. The instantaneous energy of the enhanced signal can be calculated as the signal energy.
[0083] S404: Select the target enhancement signal from the enhancement signals corresponding to the candidate directions based on the signal strength.
[0084] S405, determine the candidate direction corresponding to the target enhancement signal as the sound source direction.
[0085] In some embodiments, the enhanced signal with the highest signal intensity can be selected as the target enhanced signal from the enhanced signals corresponding to the candidate directions, and the candidate direction corresponding to the target enhanced signal can be taken as the sound source direction. By determining the candidate direction corresponding to the enhanced signal with the highest signal intensity as the sound source direction, noise interference can be suppressed and the accuracy of sound source localization can be improved.
[0086] In some embodiments, after determining the direction of the sound source, the direction of the sound source can be displayed, thereby visualizing the location of the sound source direction and allowing the driver to clearly identify the direction of the sound source. Optionally, the direction of the sound source can be displayed on an in-vehicle display screen. The direction of the sound source can also be displayed on a terminal device that interacts with the vehicle.
[0087] In some embodiments, the sound source type can also be displayed. The sound source type is displayed by identifying and outputting the sound source type. The sound source type includes, but is not limited to, natural sound sources, traffic-related sound sources, and industrial facility sound sources. The sound source type can be determined by identifying the enhanced signal and then displayed.
[0088] The sound source localization method provided in the embodiments of this disclosure determines the signal strength of the enhanced signal corresponding to the candidate direction, and determines the sound source direction based on the signal strength. The candidate direction corresponding to the enhanced signal with the largest signal strength is determined as the sound source direction, which can suppress noise interference and improve the accuracy of sound source localization.
[0089] Figure 5 This is a flowchart illustrating a sound source localization method according to an exemplary embodiment, such as... Figure 5As shown, the sound source localization method of this disclosure includes the following steps: S501, determine multiple candidate directions preset for the sound source, wherein the candidate directions include multiple directions used to represent the potential directions of the sound source.
[0090] S502, based on the first projection position of different microphones in the candidate direction, performs time delay compensation on the signal collected by the microphone to obtain the enhanced signal corresponding to the candidate direction.
[0091] S503 determines the sound source direction based on the signal intensity of the enhanced signal corresponding to the candidate direction.
[0092] The relevant content of steps S501-S503 can be found in the above embodiments, and will not be repeated here.
[0093] S504 controls the vehicle to avoid the sound source based on its direction.
[0094] It should be noted that when a vehicle is in autonomous or assisted driving mode, after determining the direction of the sound source, the vehicle's behavior can be adjusted to avoid the sound source in order to prevent accidents. This can help the vehicle adapt to complex traffic environments and improve driving safety.
[0095] In some embodiments, when controlling the vehicle to avoid a sound source, an avoidance strategy matching the sound source can be selected from preset avoidance strategies. Optionally, the vehicle's avoidance strategy can be determined based on the type of sound source.
[0096] In some embodiments, the types of sound sources include, but are not limited to, natural sound sources, traffic-related sound sources, and industrial facility sound sources. The type of sound source can be determined by feature extraction and analysis of the acquired signals.
[0097] In some embodiments, in order to improve the accuracy of signal analysis and avoid noise interference, feature information can be extracted from the target enhancement signal corresponding to the direction of the sound source, so that the type of sound source can be determined based on the extracted feature information.
[0098] In some embodiments, sound source types can be identified based on extracted feature information, and avoidance strategies can be determined according to the identified sound source types. Optionally, a set of sound source types can be predefined, wherein each sound source type in the set has its own avoidance strategy.
[0099] In other words, by determining whether the identified sound source type belongs to a preset set of sound source types, and in response to the identified sound source type belonging to the preset set of sound source types, the vehicle's avoidance strategy is determined based on the type of sound source.
[0100] In some embodiments, in response to the identified sound source type not belonging to a preset set of sound source types, the driver can be prompted to control the vehicle to avoid the obstacle.
[0101] In some embodiments, after determining the avoidance strategy, the vehicle can be controlled to avoid the sound source based on the avoidance strategy. The avoidance strategy includes, but is not limited to: giving way quickly, preventing collisions, and avoiding secondary collisions.
[0102] For example, if the sound source is identified as being from an emergency vehicle, a quick yielding strategy can be implemented to ensure the efficient passage of emergency vehicles.
[0103] The sound source localization method provided in the embodiments of this disclosure, after determining the direction of the sound source, can determine an avoidance strategy based on the type of sound source, and control the vehicle to avoid the sound source according to the avoidance strategy, so as to adjust the vehicle behavior to avoid the occurrence of accidents and improve driving safety.
[0104] Figure 6 This is a block diagram illustrating a sound source localization device according to an exemplary embodiment. (Refer to...) Figure 6 The sound source localization device 600 of this disclosure includes: a determination module 601, a compensation module 602 and a localization module 603.
[0105] The determining module 601 is used to determine multiple candidate directions preset for the sound source, wherein the candidate directions include multiple directions used to represent the potential directions of the sound source; The compensation module 602 is used to perform time delay compensation on the signal collected by the microphone according to the first projection position of different microphones in the candidate direction, so as to obtain the enhanced signal corresponding to the candidate direction; The positioning module 603 is used to determine the direction of the sound source based on the signal strength of the enhanced signal corresponding to the candidate direction.
[0106] In one embodiment of this disclosure, the compensation module 602 is further configured to: determine a reference position corresponding to the candidate direction; determine the signal delay corresponding to the microphone based on the first projection position of the microphone in the candidate direction and the reference position; and perform delay compensation on the signal collected by the microphone based on the signal delay corresponding to the microphone to obtain an enhanced signal corresponding to the candidate direction.
[0107] In one embodiment of this disclosure, the compensation module 602 is further configured to: determine a second projection position of the reference position in the candidate direction; and determine the signal delay corresponding to the microphone based on the first projection position and the second projection position.
[0108] In one embodiment of this disclosure, the compensation module 602 is further configured to: determine the propagation distance difference between the microphone and the reference position based on the first projection position and the second projection position; and determine the signal delay corresponding to the microphone based on the propagation distance difference and the signal propagation speed.
[0109] In one embodiment of this disclosure, the compensation module 602 is further configured to: determine a reference microphone corresponding to the candidate direction from among the different microphones based on the projection positions of the different microphones in the candidate direction, wherein the location of the reference microphone is a reference position corresponding to the candidate direction.
[0110] In one embodiment of this disclosure, the compensation module 602 is further configured to: determine the projection distance of different microphones in the candidate direction based on the projection positions of different microphones in the candidate direction; and determine the reference microphone corresponding to the candidate direction based on the projection distance of different microphones in the candidate direction.
[0111] In one embodiment of this disclosure, the compensation module 602 is further configured to: perform time delay compensation on the signal collected by the microphone based on the signal delay corresponding to the microphone, to obtain a compensation signal for the microphone; and perform signal fusion on the compensation signal of the microphone to obtain an enhanced signal corresponding to the candidate direction.
[0112] In one embodiment of this disclosure, the positioning module 603 is further configured to: control the vehicle to avoid the sound source based on the direction of the sound source.
[0113] In one embodiment of this disclosure, the positioning module 603 is further configured to: determine the vehicle's avoidance strategy based on the type of sound source; and control the vehicle to avoid the sound source based on the avoidance strategy.
[0114] In one embodiment of this disclosure, the positioning module 603 is further configured to: extract feature information from the target enhancement signal corresponding to the direction of the sound source; identify the sound source type based on the extracted feature information; and determine the vehicle avoidance strategy according to the type of the sound source in response to the identified sound source type belonging to a preset sound source type set.
[0115] In one embodiment of this disclosure, the positioning module 603 is further configured to: display the direction of the sound source; identify the sound source type of the sound source and output the sound source type of the sound source.
[0116] In one embodiment of this disclosure, the positioning module 603 is further configured to: extract the signal intensity of the enhancement signal corresponding to the candidate direction, wherein the signal intensity includes at least one of signal amplitude and signal energy; select a target enhancement signal from the enhancement signals corresponding to the candidate direction based on the signal intensity; and determine the candidate direction corresponding to the target enhancement signal as the sound source direction.
[0117] In one embodiment of this disclosure, the compensation module 602 is further configured to: determine a delay compensation filter corresponding to the microphone based on the signal delay corresponding to the microphone; and perform delay compensation on the signal collected by the microphone through the delay compensation filter to obtain a compensation signal corresponding to the microphone.
[0118] In one embodiment of this disclosure, the compensation module 602 is further configured to: determine the coordinate information of the microphone; and determine the first projection position of the microphone in the candidate direction based on the azimuth angle and coordinate information corresponding to the candidate direction.
[0119] The sound source localization device provided in the embodiments of this disclosure determines multiple preset candidate directions of the sound source, and determines the signal delay of the microphone based on the first projection position of different microphones in the candidate directions. It then compensates for the signal delay of the microphone-collected signals to obtain an enhanced signal corresponding to the candidate directions. Further, based on the signal strength of the enhanced signal corresponding to the candidate directions, the sound source direction can be determined. Therefore, this solution can achieve omnidirectional sound source direction identification, improving the comprehensiveness of sound source localization. By enhancing the signal collected by the microphones, the signal strength can be increased, improving the anti-interference capability of sound source localization based on signal strength, thereby improving the accuracy of sound source localization. Accurate sound source direction identification can help drivers or autonomous driving systems to avoid obstacles more quickly and accurately, improving driving safety.
[0120] Figure 7 This is a block diagram illustrating a vehicle according to an exemplary embodiment. For example, vehicle 700 can be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. Vehicle 700 can be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.
[0121] Reference Figure 7 The vehicle 700 may include various subsystems, such as an infotainment system 701, a perception system 702, a decision control system 703, a drive system 704, and a computing platform 705. The vehicle 700 may also include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and each component of the vehicle 700 can be interconnected via wired or wireless means.
[0122] In some embodiments, the infotainment system 701 may include a communication system, an entertainment system, and a navigation system, etc.
[0123] The perception system 702 may include several sensors for sensing information about the environment surrounding the vehicle 700. For example, the perception system 702 may include a global positioning system (which may be a GPS system, a BeiDou system, or another positioning system), an inertial measurement unit (IMU), a lidar, a millimeter-wave radar, an ultrasonic radar, and a camera device.
[0124] The decision control system 703 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.
[0125] The drive system 704 may include components that provide powered motion to the vehicle 700. In one embodiment, the drive system 704 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of internal combustion engines, electric motors, and compressed air engines. The engine is capable of converting energy provided by the energy source into mechanical energy.
[0126] Some or all of the functions of vehicle 700 are controlled by computing platform 705. Computing platform 705 may include at least one processor 751 and memory 752, and processor 751 may execute instructions 753 stored in memory 752.
[0127] Processor 751 can be any conventional processor, such as a commercially available CPU. Processors may also include graphics processing units (GPUs), field-programmable gate arrays (FPGAs), systems-on-chips (SoCs), application-specific integrated circuits (ASICs), or combinations thereof.
[0128] The memory 752 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.
[0129] In addition to instruction 753, memory 752 can also store data, such as road maps, route information, vehicle position, direction, speed, and other data. The data stored in memory 752 can be used by computing platform 705.
[0130] In this embodiment of the disclosure, processor 751 may execute instruction 753 to implement all or part of the steps of the sound source localization method provided in this disclosure.
[0131] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the steps of the sound source localization method provided in this disclosure.
[0132] Alternatively, the computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.
[0133] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.
[0134] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A method of acoustic source localization, the method comprising: The method comprises: determining a plurality of candidate directions of a sound source preset, wherein the candidate directions comprise a plurality of directions for representing potential directions of the sound source; performing time delay compensation on signals collected by microphones according to first projection positions of the different microphones in the candidate directions, to obtain enhanced signals corresponding to the candidate directions; determining a sound source direction based on signal strengths of the enhanced signals corresponding to the candidate directions.
2. The method of claim 1, wherein, The performing time delay compensation on the signals collected by the microphones according to the first projection positions of the different microphones in the candidate directions, to obtain the enhanced signals corresponding to the candidate directions, comprises: determining a reference position corresponding to the candidate direction; determining signal time delays corresponding to the microphones according to the first projection positions of the microphones in the candidate directions and the reference position; performing time delay compensation on the signals collected by the microphones based on the signal time delays corresponding to the microphones, to obtain the enhanced signals corresponding to the candidate directions.
3. The method of claim 2, wherein, The determining the signal time delays corresponding to the microphones according to the first projection positions of the microphones in the candidate directions and the reference position, comprises: determining a second projection position of the reference position in the candidate direction; determining the signal time delays corresponding to the microphones according to the first projection position and the second projection position.
4. The method of claim 3, wherein, The determining the signal time delays corresponding to the microphones according to the first projection position and the second projection position, comprises: determining a propagation distance difference of the microphones relative to the reference position according to the first projection position and the second projection position; determining the signal time delays corresponding to the microphones according to the propagation distance difference and a signal propagation speed.
5. The method of claim 2, wherein, The determining the reference position corresponding to the candidate direction, comprises: determining a reference microphone corresponding to the candidate direction from the different microphones according to the projection positions of the different microphones in the candidate direction, wherein a position of the reference microphone is the reference position corresponding to the candidate direction.
6. The method of claim 5, wherein, The determining the reference microphone corresponding to the candidate direction from the different microphones according to the projection positions of the different microphones in the candidate direction, comprises: determining projection distances of the different microphones in the candidate direction according to the projection positions of the different microphones in the candidate direction; determining the reference microphone corresponding to the candidate direction according to the projection distances of the different microphones in the candidate direction.
7. The method according to any one of claims 2-6, characterized in that, The performing time delay compensation on the signals collected by the microphones based on the signal time delays corresponding to the microphones, to obtain the enhanced signals corresponding to the candidate directions, comprises: performing time delay compensation on the signals collected by the microphones based on the signal time delays corresponding to the microphones, to obtain compensation signals of the microphones; performing signal fusion on the compensation signals of the microphones, to obtain the enhanced signals corresponding to the candidate directions.
8. The method according to any one of claims 1-6, characterized in that, After the determining the sound source direction, the method further comprises: controlling a vehicle to avoid the sound source according to the sound source direction.
9. The method of claim 8, wherein, The controlling the vehicle to avoid the sound source according to the sound source direction, comprises: determining an avoidance strategy of the vehicle according to a type of the sound source; Control the vehicle to avoid the sound source based on the avoidance strategy.
10. The method of claim 9, wherein, The avoidance strategy of the vehicle is determined according to the type of the sound source, including: Feature information extraction is performed on the target enhanced signal corresponding to the sound source direction; Sound source type identification is performed on the sound source based on the extracted feature information; In response to the identified sound source type belonging to a preset sound source type set, the avoidance strategy of the vehicle is determined according to the type of the sound source.
11. The method according to any one of claims 1-6, characterized in that, After determining the sound source direction, at least one of the following operations is further included: The sound source direction is displayed; The sound source type of the sound source is identified, and the sound source type of the sound source is output.
12. The method of any one of claims 1-6, wherein, The determination of the sound source direction based on the signal intensity of the enhanced signal corresponding to the candidate direction includes: Signal intensity extraction is performed on the enhanced signal corresponding to the candidate direction, wherein the signal intensity includes at least one of signal amplitude and signal energy; From the enhanced signal corresponding to the candidate direction, a target enhanced signal is selected according to the signal intensity; The candidate direction corresponding to the target enhanced signal is determined as the sound source direction.
13. The method of any one of claims 1-6, wherein, The enhanced signal corresponding to the candidate direction is obtained by performing time delay compensation on the signal collected by the microphone based on the signal time delay corresponding to the microphone, including: A time delay compensation filter corresponding to the microphone is determined according to the signal time delay corresponding to the microphone; The time delay compensation filter is used to perform time delay compensation on the signal collected by the microphone to obtain a compensation signal corresponding to the microphone.
14. The method of any one of claims 1-6, wherein, The determination process of the first projection position includes: Coordinate information of the microphone is determined; The first projection position of the microphone in the candidate direction is determined according to the azimuth angle corresponding to the candidate direction and the coordinate information.
15. A sound source positioning apparatus characterized by comprising: The device includes: A determination module is configured to determine a plurality of candidate directions of a sound source, wherein the candidate directions include a plurality of potential directions of the sound source; A compensation module is configured to perform time delay compensation on the signal collected by the microphone based on the first projection position of the microphone in the candidate direction to obtain an enhanced signal corresponding to the candidate direction; A positioning module is configured to determine a sound source direction based on the signal intensity of the enhanced signal corresponding to the candidate direction.
16. A vehicle characterized by comprising: It includes: A processor; A memory for storing processor-executable instructions; The processor is configured to: Implement the steps of the method of any one of claims 1-14.
17. A computer-readable storage medium having stored thereon computer program instructions, wherein, The program instructions are executed by the processor to implement the steps of the method of any one of claims 1-14.