Map generation method, vehicle control method, device, vehicle, and storage medium

Abstract

Embodiments of the present application provide a map generation method, a vehicle control method, a device, a vehicle and a storage medium. The method comprises: in the process of driving a first vehicle, acquiring an initial sound wave signal set collected by a microphone array at different driving positions; processing the initial sound wave signal set to determine a plurality of first sound wave signals; identifying the plurality of second sound wave signals to determine a plurality of static sound source groups in the environment in which the first vehicle is located, and constructing a static acoustic map of the environment in which the first vehicle is located based on sound source information of each static sound source group; positioning each third sound wave signal to obtain a dynamic sound source to which each third sound wave signal belongs, and determine a motion trajectory of each dynamic sound source; and fusing the static acoustic map and the motion trajectory of each dynamic sound source to obtain map data of the environment in which the first vehicle is located, thereby improving non-line-of-sight perception capability and achieving detection of an occluded target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle control, and in particular to a map generation method, a vehicle control method, a device, a vehicle and a storage medium. BACKGROUND

[0002] The environmental perception technology refers to a process in which a vehicle uses sensors and algorithms to detect, identify and model the surrounding environment, aiming to provide accurate and real-time data basis for decision control tasks such as path planning, obstacle avoidance and risk prediction.

[0003] The existing environmental perception scheme mainly relies on two types of sensors: visual sensors (such as cameras, infrared sensors, etc.) and radio frequency sensors (such as laser radars, millimeter wave radars, etc.). Among them, for the scheme of visual sensors, the targets (such as obstacles) are detected and identified through the collected images, but this way cannot perceive the occluded targets, and is easily affected by the environmental lighting conditions. For the scheme of radio frequency sensors, the targets are detected and identified by actively transmitting and receiving signals, which has strong anti-interference ability, but because the signal cannot penetrate solid obstacles, it is also difficult to perceive the occluded targets, and there is a blind area of perception. In addition, this way needs to actively transmit signals, resulting in high power consumption. SUMMARY

[0004] The embodiments of the present application provide a map generation method, a vehicle control method, a device, a vehicle and a storage medium, which improve the non-line-of-sight perception capability, can detect occluded targets, realize the reconstruction of the environmental static structure and the tracking of dynamic targets, and adopt a passive perception mechanism, which can reduce power consumption.

[0005] In a first aspect, the embodiments of the present application provide a map generation method applied to a first vehicle, a microphone array is arranged on the first vehicle, and the method comprises: acquiring an initial sound wave signal set collected by the microphone array at different driving positions in the driving process of the first vehicle; processing the initial sound wave signal set to determine a plurality of first sound wave signals, the plurality of first sound wave signals comprising a plurality of second sound wave signals and a plurality of third sound wave signals, the type of the sound source to which the second sound wave signal belongs is a static sound source type, and the type of the sound source to which the third sound wave signal belongs is a dynamic sound source type; identifying the plurality of second sound wave signals to determine a plurality of static sound source groups in the environment in which the first vehicle is located, and constructing a static acoustic map of the environment in which the first vehicle is located based on the sound source information of each static sound source group; positioning each third sound wave signal to obtain a dynamic sound source to which each third sound wave signal belongs, and determine a motion trajectory of each dynamic sound source; and fusing the static acoustic map and the motion trajectory of each dynamic sound source to obtain map data of the environment in which the first vehicle is located.

[0006] In a second aspect, the embodiments of the present application provide a vehicle control method, applied to a first vehicle, the method comprising: obtaining map data of an environment in which the first vehicle is located, the map data being generated according to the map generation method of the first aspect; generating a first auxiliary driving instruction according to the map data; and executing the first auxiliary driving instruction to control the first vehicle to travel.

[0007] In a third aspect, the embodiments of the present application provide a map generation device, configured in a first vehicle, the first vehicle being provided with a microphone array, the device comprising:

[0008] a signal obtaining module, configured to obtain a set of initial sound wave signals collected by the microphone array at different driving positions of the first vehicle during driving of the first vehicle;

[0009] a determining module, configured to process the set of initial sound wave signals to determine a plurality of first sound wave signals, the plurality of first sound wave signals including a plurality of second sound wave signals and a plurality of third sound wave signals, the type of a sound source to which the second sound wave signals belong being a static sound source type, and the type of a sound source to which the third sound wave signals belong being a dynamic sound source type;

[0010] a static map construction module, configured to identify the plurality of second sound wave signals to determine a plurality of static sound source groups in the environment in which the first vehicle is located, and construct a static acoustic map of the environment in which the first vehicle is located based on sound source information of each static sound source group;

[0011] a dynamic trajectory generation module, configured to locate each third sound wave signal to obtain a dynamic sound source to which each third sound wave signal belongs, and determine a motion trajectory of each dynamic sound source;

[0012] a fusion module, configured to fuse the static acoustic map and the motion trajectories of the dynamic sound sources to obtain the map data of the environment in which the first vehicle is located.

[0013] In a fourth aspect, the embodiments of the present application provide a vehicle control device, configured in a first vehicle, the device comprising:

[0014] a data obtaining module, configured to obtain map data of an environment in which the first vehicle is located;

[0015] an instruction generation module, configured to generate a first auxiliary driving instruction according to the map data;

[0016] a control module, configured to execute the first auxiliary driving instruction to control the first vehicle to travel.

[0017] In a fifth aspect, the embodiments of the present application provide a vehicle, comprising: a memory, a processor;

[0018] the memory stores computer execution instructions;

[0019] The processor executes computer-executed instructions stored in the memory, so that the processor executes the first aspect and / or various possible implementation manners of the first aspect as above, or executes the second aspect and / or various possible implementation manners of the second aspect as above.

[0020] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, and the computer-readable storage medium stores computer-executed instructions. The computer-executed instructions are executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect as above, or to implement the second aspect and / or various possible implementation manners of the second aspect as above.

[0021] In a seventh aspect, an embodiment of the present application provides a computer program product, which includes a computer program. The computer program is executed by a processor to implement the first aspect and / or various possible implementation manners of the first aspect as above, or to implement the second aspect and / or various possible implementation manners of the second aspect as above.

[0022] In the embodiment of the present application, during the driving of the vehicle, an initial sound wave signal set is collected by a microphone array, the initial sound wave signal set is processed, and a plurality of first sound wave signals are determined. For a second sound wave signal belonging to a static sound source in the plurality of first sound wave signals, the second sound wave signal is clustered and optimized, and a static acoustic map is constructed. For a third sound wave signal belonging to a dynamic sound source in the plurality of first sound wave signals, the third sound wave signal is tracked and identified, and a motion trajectory of the dynamic sound source is generated. The static acoustic map and the motion trajectory of each dynamic sound source are fused to obtain map data of an environment in which the first vehicle is located. In this way, by using the diffraction characteristics of sound waves, the target blocked by buildings or other obstacles can be detected, the visual field blind area of the optical and radio frequency sensors is effectively compensated, the non-line-of-sight perception capability is improved, and thus the coverage range and early warning capability of the entire perception system are improved. Furthermore, the sound wave signals in the environment in which the vehicle is located are passively collected, a detection signal does not need to be actively sent out, the complexity of data processing can be reduced, the processing efficiency can be improved, the power consumption can be reduced, electromagnetic interference can be avoided, and the accuracy of the generated map data can be improved. In addition, a high-precision static environment map and a dynamic target trajectory can be synchronously output, and a more comprehensive and complete environment cognition model is constructed. BRIEF DESCRIPTION OF DRAWINGS

[0023] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0024] Figure 1 A schematic diagram of an application scenario of a map generation method according to an embodiment of the present application;

[0025] Figure 2FIG. 1 is a schematic diagram of an application scenario of another map generation method according to an embodiment of the present application;

[0026] Figure 3 FIG. 2 is a schematic diagram of a flow of a map generation method according to an embodiment of the present application;

[0027] Figure 4 FIG. 3 is a schematic diagram of another flow of a map generation method according to an embodiment of the present application;

[0028] Figure 5 FIG. 4 is a schematic diagram of a third flow of a map generation method according to an embodiment of the present application;

[0029] Figure 6 FIG. 5 is a schematic diagram of a fourth flow of a map generation method according to an embodiment of the present application;

[0030] Figure 7 FIG. 6 is a schematic diagram of a fifth flow of a map generation method according to an embodiment of the present application;

[0031] Figure 8 FIG. 7 is a schematic diagram of a sixth flow of a map generation method according to an embodiment of the present application;

[0032] Figure 9 FIG. 8 is a schematic diagram of a seventh flow of a map generation method according to an embodiment of the present application;

[0033] Figure 10 FIG. 9 is a schematic diagram of an eighth flow of a map generation method according to an embodiment of the present application;

[0034] Figure 11 FIG. 10 is a schematic diagram of a flow of a vehicle control method according to an embodiment of the present application;

[0035] Figure 12 FIG. 11 is a schematic diagram of a structure of a map generation device according to an embodiment of the present application;

[0036] Figure 13 FIG. 12 is a schematic diagram of a structure of a vehicle control device according to an embodiment of the present application;

[0037] Figure 14 FIG. 13 is a schematic diagram of a structure of a vehicle according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] The embodiments will be described in detail below with reference to the drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following embodiments do not represent all implementations consistent with the present application. Rather, they are merely examples of systems and methods consistent with some aspects of the present application as detailed in the appended claims.

[0039] It should be noted that the brief description of the terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of the application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0040] The terms "first", "second", "third" and the like in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit the specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchanged under appropriate circumstances.

[0041] The terms "include" and "have" and any variations thereof are intended to cover but not exclusive inclusion, for example, a product or device including a series of components does not have to be limited to all components clearly listed, but can include other components not clearly listed or inherent to these products or devices.

[0042] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware or / and software code capable of performing functions related to the element.

[0043] The environmental perception technology refers to the process in which the vehicle uses sensors and algorithms to detect, identify and model its surrounding environment, aiming to provide accurate and real-time data basis for path planning, obstacle avoidance and risk prediction decision control tasks.

[0044] The existing environmental perception scheme mainly relies on two types of sensors: visual sensors (such as cameras, infrared sensors, etc.) and radio frequency sensors (such as laser radars, millimeter wave radars, etc.). Among them, for the scheme of visual range sensor, target (such as obstacle) detection and identification are performed through the collected images, but this way cannot perceive the occluded target, and is easily affected by the environmental lighting conditions. For the scheme of radio frequency sensor, active emission and reception of signals are used to realize the detection and identification of the target, which has strong anti-interference ability, but because the signal cannot penetrate solid obstacles, it is also difficult to perceive the occluded target, and there is a blind area of perception. In addition, this way needs to actively emit signals, resulting in high power consumption.

[0045] In summary, the existing environmental perception scheme lacks effective non-line-of-sight perception capability and cannot detect occluded targets. In addition, due to the active detection mechanism of the sensor, the power consumption is large and the cost is high.

[0046] Therefore, the embodiment of the present application provides a map generation method. In the driving process of a vehicle, an initial sound wave signal set is collected by a microphone array, the initial sound wave signal set is processed, and a plurality of first sound wave signals are determined. For a second sound wave signal belonging to a static sound source in the plurality of first sound wave signals, the second sound wave signal is clustered and optimized to construct a static acoustic map. For a third sound wave signal belonging to a dynamic sound source in the plurality of first sound wave signals, the third sound wave signal is tracked and identified to generate a motion trajectory of the dynamic sound source. The static acoustic map and the motion trajectory of each dynamic sound source are fused to obtain map data of an environment in which the first vehicle is located. In this way, the diffraction characteristics of sound waves can be used to detect targets blocked by buildings or other obstacles, effectively compensate for the visual blind area of optical and radio frequency sensors, improve non-line-of-sight perception capability, and thus improve the coverage range and early warning capability of the entire perception system.

[0047] Before introducing the map generation method provided by the embodiment of the present application, the application scenario of the map generation method is described.

[0048] Figure 1 An application scenario of a map generation method according to an embodiment of the present application is shown in the figure.

[0049] The map generation method provided by the embodiment of the present application is applied to a vehicle. As shown in Figure 1 For example, in the driving process of the vehicle 10, the microphone array 101 arranged on the vehicle 10 can continuously collect mixed sound wave signals at different positions. Sound source positioning and separation are performed on the mixed sound wave signals to obtain a plurality of effective sound wave signals. By analyzing and identifying the effective sound wave signals, a plurality of sound sources in the environment in which the vehicle 10 is located are determined, such as sound source a, sound source b, sound source c, sound source d, and sound source e. The sound source can be a static sound source, such as a tree in a separation belt, a signal lamp, or a building. The sound source can also be a dynamic sound source, such as another vehicle, a pedestrian, or the like. Based on this, a static acoustic map containing static sound sources is constructed, and the static acoustic map and the motion trajectory of the dynamic sound source are fused to obtain map data representing the environment in which the vehicle 10 is located. The vehicle 10 can implement vehicle positioning, path planning, driving intention prediction, and other auxiliary driving functions based on the generated map data.

[0050] It should be noted that the above-mentioned sound source can refer to a detection object that can generate a sound wave signal itself, such as another vehicle, a pedestrian, a signal lamp, an air conditioner, a transformer, or the like in the environment in which the vehicle is located. The sound source can also refer to a detection object that cannot generate a sound wave signal itself, but can reflect or refract the sound wave signal emitted by another sound source, such as an obstacle, a building, a separation belt, or the like in the environment in which the vehicle is located.

[0051] The map generation method provided by the embodiment of the present application can also be applied to a vehicle-to-vehicle system composed of a plurality of vehicles. As shown inFigure 2 As shown in the figure, taking vehicle 10 and vehicle 20 as an example, when vehicle 20 drives to the vicinity of vehicle 10, i.e., vehicle 20 is in the same environment as vehicle 10, vehicle 10 can share the generated map data to vehicle 20, so that vehicle 20 can also realize vehicle positioning, path planning, driving intention prediction and other auxiliary driving functions based on the map data.

[0052] Of course, vehicle 10 can also upload the generated map data to the cloud, and when other vehicles (such as vehicle 20) drive into the corresponding area, the map data corresponding to the area can be obtained from the cloud, and then vehicle positioning, path planning, driving intention prediction and other auxiliary driving functions can be realized based on the map data.

[0053] The map generation method provided by the embodiments of the present application will be described in detail below in combination with the drawings and application scenarios.

[0054] Figure 3 A flowchart of a map generation method provided by an embodiment of the present application is shown in the figure. Figure 3 As shown in the figure, the map generation method can include the following steps:

[0055] S301, in the driving process of the first vehicle, a set of initial sound wave signals collected by a microphone array at different driving positions is obtained.

[0056] Among the set of initial sound wave signals, there are multiple groups of initial sound wave signals, and a group of initial sound wave signals includes initial sound wave signals collected by each microphone in the microphone array. The initial sound wave signal collected by one microphone is a mixed sound wave signal composed of multiple sound wave signals from different sound sources in the environment where the first vehicle is located.

[0057] In some examples, one microphone array can be provided on the first vehicle, or multiple microphone arrays can be provided. The microphone array can be arranged inside the first vehicle, or arranged outside the first vehicle. The microphone array can be arranged on the front side of the first vehicle, or arranged on the rear side of the first vehicle. The number and arrangement position of the microphone array in the first vehicle are not limited in the embodiments of the present application.

[0058] The microphone array is composed of multiple microphones, i.e., multiple microphones arranged in a specific geometric shape (such as a uniform circular array, a spiral array). The multiple microphones can be arranged at equal intervals, or can be arranged at unequal intervals. The number of microphones in the microphone array and the distance between each microphone are not limited in the embodiments of the present application.

[0059] Exemplarily, the microphone can be a surface wave microphone. For convenience of description, the microphone array can also be referred to as a surface wave microphone array. The surface wave microphone is used to pick up a sound wave signal propagating through a medium surface (such as the ground, a building surface, etc.). During vehicle driving, the sound wave signal in the environment where the vehicle is located is collected by the surface wave microphone array, which can effectively suppress the environmental noise propagating in the air, thereby enhancing the capture capability of the effective vibration signal generated by other vehicles or pedestrians and conducted through the medium surface, and improving the signal-to-noise ratio.

[0060] In a specific implementation, the microphone array collects the mixed sound wave signal at a set sampling frequency, and the collected mixed sound wave signal is a continuous signal.

[0061] In some embodiments, after obtaining the initial sound wave signal set collected by the microphone array at different driving positions, the mixed sound wave signal collected by each microphone is preprocessed for each group of initial sound wave signals to suppress environmental noise and reduce the influence of multipath effect. The process of preprocessing the initial sound wave signal can be referred to as the description of the preset processing process below, which will not be described here.

[0062] S302, the initial sound wave signal set is processed to determine a plurality of first sound wave signals, and the plurality of first sound wave signals are sound wave signals originating from different sound sources in the environment of the first vehicle.

[0063] Among them, the first sound wave signal is an effective sound wave signal separated from a group of initial sound wave signals collected by the microphone array of the first vehicle.

[0064] In the embodiments of the present application, the sound sources identified by the vehicle can be divided into two categories. One is a static sound source, which can be a sound source object that can generate sound wave signals itself. For example, a signal light, an air conditioner outdoor unit, a transformer, a sound box, etc. The static sound source can also be a sound source object that cannot generate sound wave signals itself, but can reflect or refract sound wave signals emitted by other sound sources. For example, obstacles on the road, building surfaces, walls, barriers, etc. The other is a dynamic sound source, such as other vehicles, pedestrians, etc.

[0065] In some examples, the sound sources identified by the vehicle can also include a self sound source (also referred to as a general self sound source), which refers to the sound generated by the vehicle (such as the first vehicle) itself. For example, the motor noise of the first vehicle, the air conditioner noise, the sound output by the in-vehicle audio system, etc.

[0066] In some examples, the sound sources identified by the vehicle can also include a specified sound source (also referred to as a special sound source), which refers to a sound source with specific and identifiable time-frequency characteristics. For example, ambulance siren, alarm, vehicle horn sound, brake sound, etc.

[0067] That is, the sound source type of the sound source to which the sound wave signal belongs can include a static sound source type, a dynamic sound source type, a self sound source type, and a designated sound source type. In the embodiments of the present application, the plurality of first sound wave signals separated from the initial sound wave signal set collected by the microphone array can include at least one second sound wave signal and at least one third sound wave signal. Among them, the second sound wave signal is a static sound source type of the sound source to which it belongs (i.e., the second sound wave signal is the above-mentioned static sound source). The third sound wave signal is a dynamic sound source type of the sound source to which it belongs (i.e., the third sound wave signal is the above-mentioned dynamic sound source).

[0068] In some embodiments, processing the initial sound wave signal set to determine the plurality of first sound wave signals can include: performing sound source positioning and separation on each group of initial sound wave signals to determine the plurality of first sound wave signals, and determining the sound source type of the sound source to which each first sound wave signal belongs.

[0069] Optionally, after preprocessing the initial sound wave signal, the preprocessed initial sound wave signal is subjected to sound source positioning and separation to determine the plurality of first sound wave signals. In this way, the plurality of effective sound wave signals can be more accurately separated, and the accuracy of sound source identification can be improved in combination with subsequent steps.

[0070] It should be noted that the process of determining the plurality of first sound wave signals can refer to the sound source positioning and separation process described below, which will not be described here.

[0071] In some embodiments, the plurality of first sound wave signals includes a fourth sound wave signal of a self sound source type. In this regard, after processing the initial sound wave signal set to determine the plurality of first sound wave signals, the method further includes: removing the fourth sound wave signal from the plurality of first sound wave signals. In this way, the sound signal generated by the first vehicle itself can be avoided to cause interference, and the accuracy of sound source identification in the environment where the first vehicle is located can be improved.

[0072] S303, identifying the plurality of second sound wave signals to determine a plurality of static sound source groups in the environment where the first vehicle is located, and constructing a static acoustic map of the environment where the first vehicle is located based on the sound source information of each static sound source group.

[0073] The second sound wave signals are of a static sound source type. It can be understood that the second sound wave signals can include sound wave signals from the same sound source, or sound wave signals from different sound sources. In this regard, the second sound wave signals are clustered and identified, and a plurality of static sound source groups are determined according to the clustering results. A static sound source group can include a static sound source that generates corresponding sound wave signals, or a potential static sound source that reflects and refracts sound wave signals emitted by the static sound source, i.e., a potential static sound source related to the static sound source. Therefore, the static sound source group can represent a group of acoustic reflecting units including the static sound source and the potential static sound source related to the static sound source, rather than a single sound source.

[0074] For example, an air conditioner outdoor unit (i.e., a static sound source) outside a building on the roadside and a building wall (i.e., a potential static sound source) form a static sound source group.

[0075] The static acoustic map is used to indicate a plurality of static sound source groups in an environment in which the first vehicle is located. The static acoustic map can structurally represent the spatial distribution and characteristics of acoustic beacons (i.e., static sound source groups) in the environment in which the first vehicle is located, i.e., a characteristic acoustic beacon database.

[0076] It should be noted that the static acoustic map can include sound source information corresponding to each static sound source group, i.e., sound source information of each static sound source and potential static sound source. The sound source information at least includes spatial position information (i.e., spatial characteristics) and acoustic characteristics (i.e., acoustic fingerprints) of the static sound source group. The spatial position information of the static sound source group is used to indicate the position of the static sound source group in the environment, such as coordinates, azimuth angle, and pitch angle. The acoustic characteristics (i.e., acoustic fingerprints) can be, for example, acoustic feature vectors described below.

[0077] In addition, the sound source information can also include semantic labels. The semantic labels can represent the type of the static sound source group. It should be noted that after determining the static sound source group, the type of the static sound source group is determined according to the static sound source characteristics of the static sound source group, and the corresponding semantic label is generated according to the type of the static sound source group.

[0078] It should be noted that the process of constructing the static acoustic map of the environment in which the first vehicle is located can be referred to below, and will not be described here.

[0079] In some embodiments, during the driving of the first vehicle, if a new static sound source is detected, a new static sound source is added in the static acoustic map.

[0080] In some embodiments, when the first vehicle is at a certain position, the microphone array collects initial sound wave signals at a preset sampling frequency. If a certain static sound source group is not detected at the sampling point, a confidence level is set for the static sound source group. Moreover, if the static sound source group is not detected at the next sampling point, the confidence level is updated, and the confidence level gradually decreases. When the confidence level of the static sound source group is less than a confidence threshold, it is determined that the static sound source group is invalid, and the static sound source group in the static acoustic map is deleted. In this way, the timeliness and accuracy of the static acoustic map can be maintained.

[0081] In some embodiments, after the static acoustic map is generated, a historical acoustic map is obtained according to the current position and the current time of the first vehicle. Each static sound source group in the static acoustic map is compared with each static sound source group in the historical acoustic map, and the static acoustic map is updated according to the sound source information of each static sound source group in the historical acoustic map.

[0082] Specifically, the sound source information (such as spatial features and acoustic features) of the static sound source group in the static acoustic map is similarity matched with the sound source information of the static sound source group in the historical acoustic map. If a certain static sound source group (i.e., a map point) in the static acoustic map matches a certain static sound source group (i.e., a map point) in the historical acoustic map, and the spatial position of the static sound source group matches the current pose of the first vehicle, it is determined that the first vehicle travels through the area. Enter the loop constraint, trigger a global optimization, and thus synchronously obtain an optimized globally consistent static acoustic map.

[0083] S304, positioning each third sound wave signal to obtain a dynamic sound source to which each third sound wave signal belongs, and determining a motion trajectory of each dynamic sound source.

[0084] The third sound wave signal is a sound wave signal whose sound source type is a dynamic sound source type.

[0085] It should be noted that the determination process of the motion trajectory of the dynamic sound source can refer to the dynamic sound source tracking process described below, which will not be described here.

[0086] S305, fusing the static acoustic map and the motion trajectory of each dynamic sound source to obtain map data of an environment in which the first vehicle is located.

[0087] In some embodiments, the first vehicle is also provided with a visual sensor and a radio frequency sensor. The visual sensor and the radio frequency sensor can be used to obtain visual map data and laser map data of an environment in which the first vehicle is located. The map data, the visual map data, and the laser map data of the environment in which the first vehicle is located are fused to generate environment perception data.

[0088] Optionally, the fusion method can use an extended Kalman filter or factor graph optimization to tightly couple the acoustic simultaneous localization and mapping algorithm output with inertial measurement unit, visual odometry, and other sensor data.

[0089] Optionally, the system also optionally integrates a self-supervised learning module, which automatically corrects the parameter drift of the acoustic model by comparing the consistency of the acoustic map and the visual / laser map, and improves the long-term stability and generalization ability in complex environments.

[0090] In some embodiments, the plurality of first sound wave signals includes a fifth sound wave signal of a specified sound source type; the method further includes: adding a specified label to the static sound source group in the map data when the type of the static sound source group is the specified type; and adding a specified label to the dynamic sound source in the map data when the type of the dynamic sound source is the specified type.

[0091] In addition, in the case of the identified fifth sound wave signal, the first vehicle outputs alarm information to prompt the user to avoid, thereby improving driving safety.

[0092] In the embodiments of the present application, during vehicle driving, an initial sound wave signal set is collected by a microphone array, the initial sound wave signal set is processed, and a plurality of first sound wave signals are determined. For a second sound wave signal belonging to a static sound source in the plurality of first sound wave signals, the second sound wave signal is clustered and optimized to construct a static acoustic map. For a third sound wave signal belonging to a dynamic sound source in the plurality of first sound wave signals, the third sound wave signal is tracked and identified to generate a motion trajectory of the dynamic sound source. The static acoustic map and the motion trajectory of each dynamic sound source are fused to obtain map data of an environment in which the first vehicle is located. In this way, by using the diffraction characteristics of sound waves, targets blocked by buildings or other obstacles can be detected, effectively compensating for the visual field blind area of optical and radio frequency sensors, improving non-line-of-sight perception capability, and thereby improving the coverage range and early warning capability of the entire perception system. Furthermore, the embodiments of the present application passively collect sound wave signals in the environment in which the vehicle is located, do not need to actively emit detection signals, can reduce the complexity of data processing, improve processing efficiency, and reduce power consumption, avoid electromagnetic interference, and improve the accuracy of map data generation. In addition, a high-precision static environment map and a dynamic target trajectory can be output synchronously, and a more comprehensive and complete environment cognition model is constructed.

[0093] The preprocessing process, sound source positioning and separation process, static acoustic map construction process, dynamic sound source tracking process, and static and dynamic information conflict resolution process will be described below.

[0094] 1. Preprocessing process

[0095] Figure 4A flowchart of a map generation method provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, in the map generation method, the process of preprocessing the initial sound wave signals (mixed sound wave signals) collected by each microphone can include the following steps: Figure 4

[0096] S401, filtering processing is performed on the initial sound wave signals to obtain filtered initial sound wave signals.

[0097] In the first vehicle driving process, the initial sound wave signal set is collected by the microphone array, and the initial sound wave signal collected by each microphone in the microphone array is filtered.

[0098] Specifically, the initial sound wave signal collected by the microphone is subjected to band-pass filtering processing to retain the sound wave signal of a set frequency band, and the filtered initial sound wave signal is obtained. The initial sound wave signal collected by the microphone is a continuous sound wave signal, which can be represented as , and the filtered initial sound wave signal can be represented as , where t is a continuous time variable.

[0099] S402, the filtered initial sound wave signal is subjected to multipath suppression processing to obtain a multipath-suppressed initial sound wave signal.

[0100] Specifically, the filtered initial sound wave signal is subjected to multipath suppression processing to obtain a multipath-suppressed initial sound wave signal. The multipath-suppressed initial sound wave signal can be represented as .

[0101] It should be noted that for each candidate sound source (preset sound source), the multipath components in the received signal are analyzed to construct a multipath delay fingerprint. If two or more signal components are separated from the mixed sound wave signal (i.e. the initial sound wave signal), the multipath delay fingerprint of each signal is determined, if the main path direction of the two or more signal components is close, and the multipath delay fingerprints of the two or more signal components are different, it is determined that the two or more signal components come from two different sound sources (i.e. physical sound sources).

[0102] S403, the multipath-suppressed initial sound wave signal is subjected to signal enhancement processing to obtain an enhanced initial sound wave signal.

[0103] Specifically, the multipath-suppressed initial sound wave signal is subjected to signal enhancement processing to obtain an enhanced initial sound wave signal. The enhanced initial sound wave signal can be represented as .

[0104] For example, as shown in FIG. 2, the map generation method can include the following steps: Figure 5 ​As shown, the process of performing signal enhancement processing on the initial sound wave signal after the multipath suppression processing can include the following steps:

[0105] S501, performing segmentation processing on the initial sound wave signal after the multipath suppression processing to obtain a plurality of short-time signal segments.

[0106] Specifically, according to the preset sliding window parameters, the initial sound wave signal is segmented to obtain a plurality of short-time signal segments, i.e., a discrete short-time signal segment sequence. For example, the window length (i.e., frame length) is set to 20 ms, and the overlapping time length between adjacent frames is 10 ms, so that the continuous initial sound wave signal can be segmented into a series of short-time signal segments (i.e., short-time frames), each frame containing N sampling points.

[0107] S502, for each short-time signal segment, determining the time domain feature of the short-time signal segment.

[0108] The time domain feature can include short-time energy. The short-time energy can represent the size of the signal amplitude of the short-time signal segment.

[0109] Exemplarily, the short-time energy of the short-time signal segment can be calculated according to the following formula (1).

[0110] (1)

[0111] Wherein, represents the short-time energy of the nth short-time signal segment, represents the discrete short-time signal segment sequence of the i-th microphone after the multipath suppression processing, represents the absolute index of the sampling point in the discrete short-time signal segment sequence, represents the absolute index of the starting sampling point of the nth short-time signal segment in the discrete short-time signal segment sequence, and N represents the number of sampling points contained in the nth short-time signal segment.

[0112] The time domain feature can also include the zero-crossing rate (i.e., the average zero-crossing rate). The zero-crossing rate can represent the number of times the short-time signal segment crosses zero in a unit of time.

[0113] Exemplarily, the average zero-crossing rate can be calculated according to the following formula (2).

[0114] (2)

[0115] Wherein, represents the average zero-crossing rate of the nth short-time signal segment. represents the sign function, which outputs 1 when the input is positive and -1 when the input is negative.

[0116] S503, performing signal enhancement processing on the short-time signal segment based on the time-domain feature of the short-time signal segment, to obtain an enhanced short-time signal segment.

[0117] In some examples, the signal enhancement processing is performed on the short-time signal segment to obtain an enhanced short-time signal segment, in a case that the short-time energy of the short-time signal segment is less than a preset short-time energy threshold, and the zero-crossing rate of the short-time signal segment is less than a preset zero-crossing rate threshold.

[0118] The preset short-time energy threshold can represent an energy level of the background noise. In actual application, in a case that the vehicle is in a quiet environment, the background noise is collected and the short-time energy of the background noise is determined, and the preset short-time energy threshold is determined based on the short-time energy of the background noise.

[0119] The preset zero-crossing rate threshold is used to distinguish whether the short-time signal segment (i.e., the initial sound wave signal) is a low-frequency sound event or a high-frequency sound event.

[0120] In the judgment of whether the short-time signal segment contains an effective sound event, the short-time energy of the short-time signal segment is compared with the preset short-time energy threshold. If the short-time energy of the short-time signal segment is greater than or equal to the preset short-time energy threshold, it indicates that the short-time signal segment contains an effective sound event, i.e., contains a sound event other than the background noise. If the short-time energy of the short-time signal segment is less than the preset short-time energy threshold, the zero-crossing rate of the short-time signal segment is compared with the preset zero-crossing rate threshold. If the zero-crossing rate of the short-time signal segment is less than the preset zero-crossing rate threshold, it is determined that the short-time signal segment contains an effective sound event, and the effective sound event is a low-frequency sound event.

[0121] In a case that it is determined that the short-time signal segment contains an effective sound event, and the effective sound event is a low-frequency sound event, the signal enhancement processing is performed on the short-time signal segment to obtain an enhanced short-time signal segment.

[0122] In some examples, in the signal enhancement processing on the short-time signal segment, the signal enhancement processing can be performed on the short-time signal segment according to a preset gain parameter, to obtain an enhanced short-time signal segment.

[0123] For example, the enhanced short-time signal segment can be calculated according to the following formula (3).

[0124] (3)

[0125] Wherein, The enhanced short-time signal segment is represented by x (n), The gain parameter is represented by g.

[0126] Optionally, the gain parameter can be determined based on a target amplitude and the short-time energy of the short-time signal segment.

[0127] For example, the gain parameter can be calculated according to the following formula (4).

[0128] (4)

[0129] wherein, denotes the gain parameter, denotes a target amplitude, denotes a short-term energy of the nth frame of the short-term signal segment, is a preset constant.

[0130] The target amplitude is set based on a dynamic range of the system and a signal characteristic. For example, the target amplitude can be a fixed value. For example, = 0.1 FS, FS being a full-scale amplitude of the system, so as to ensure that a strong signal is not saturated and sufficient gain is provided for a weak signal.

[0131] For example, the target amplitude is related to a long-term energy level of the signal. For example, , is a long-term average energy, which can be calculated by a first-order IIR filter or other common methods, and k is a scaling factor, so as to maintain the short-term signal segment in an ideal dynamic range.

[0132] In some examples, when performing signal enhancement processing on the short-term signal segment, the gain parameter can be smoothed, and the signal enhancement processing is performed on the short-term signal segment by using the smoothed gain parameter, so as to avoid gain mutation and prevent gain oscillation.

[0133] For example, the smoothed gain parameter can be calculated according to the following formula (5).

[0134] (5)

[0135] wherein, denotes the smoothed gain parameter, denotes the smoothed gain parameter of the previous frame of the short-term signal segment, denotes the gain parameter, denotes a smoothing factor.

[0136] By adjusting the smoothing factor, the gain variation can be controlled to be not too large. For example, when the short-term energy of the short-term signal segment increases sharply, a smaller smoothing factor is set to decay quickly.

[0137] Optionally, a gain boundary, i.e., a gain upper limit value and a gain lower limit value When the gain parameter after smoothing is less than the gain lower limit value, the gain lower limit value is taken as the gain parameter after smoothing, so as to prevent oversaturation. When the gain parameter after smoothing is less than the gain upper limit value, the gain upper limit value is taken as the gain parameter after smoothing, so as to prevent excessive noise.

[0138] In this regard, the enhanced short-time signal segment can be calculated according to the following formula (6).

[0139] (6)

[0140] wherein, denotes the enhanced short-time signal segment, denotes the gain parameter after smoothing, .

[0141] S504, superimpose the plurality of enhanced short-time signal segments to obtain a preprocessed initial sound wave signal.

[0142] wherein, the preprocessed initial sound wave signal is a continuous time-domain signal, and can be expressed as .

[0143] In the embodiments of the present application, after obtaining the initial sound wave signal set collected by the microphone array at different driving positions, the mixed sound wave signals collected by each microphone are preprocessed for each group of initial sound wave signals to suppress environmental noise and reduce the influence of multipath effect.

[0144] In some embodiments, after preprocessing the initial sound wave signal, a short-time Fourier transform is performed on the preprocessed initial sound wave signal to obtain a time-frequency spectrum; the spectral entropy of each time frame is determined, and the spectral entropy is used to represent the complexity of the preprocessed initial sound wave signal. The lower the value of the spectral entropy of the preprocessed initial sound wave signal, the simpler the initial sound wave signal, i.e., the initial sound wave signal is a noise signal. The higher the value of the spectral entropy of the preprocessed initial sound wave signal, the more complex the initial sound wave signal, i.e., the initial sound wave signal includes multiple sound wave signals.

[0145] For example, the spectral entropy of each time frame can be determined according to the following formula (7).

[0146] (7)

[0147] wherein, denotes the spectral entropy of each time frame, is the probability mass function of the power spectrum of the frame after normalization, denotes the time-frequency spectrum, , denotes the lower and upper limits of the frequency range.

[0148] To this end, after determining the spectral entropy of each time frame, the average spectral entropy of the preset time period is determined. If the average spectral entropy is less than the preset spectral entropy threshold, it indicates that the initial sound wave signal is simpler, and the active sound emitting module on the first vehicle emits sound.

[0149] The active sound emitting module emits an acoustic signal with known characteristics. For example: linear frequency modulation pulse (with good autocorrelation characteristics, conducive to echo detection and time delay estimation), pseudo-random sequence (such as m sequence) (robust to environmental disturbances, easy to separate from noise), etc. The system then simultaneously processes the environmental sound and the reflected echo of the actively emitted signal. The echo provides an additional, known acoustic beacon for assisting positioning and mapping.

[0150] 2. Sound source positioning and separation process

[0151] Specifically, the sound source positioning and separation are performed on each set of initial sound wave signals to determine a plurality of first sound wave signals.

[0152] In some examples, the set of initial sound wave signals includes initial sound wave signals collected by each microphone in the microphone array, and an initial sound wave signal collected by a microphone includes a plurality of sound wave signals originating from different sound sources. To this end, for each initial sound wave signal collected by a microphone, the initial sound wave signal is subjected to Fourier transform to obtain a time-frequency spectrum, for each time-frequency point, the spatial characteristics of the time-frequency point between the microphones are calculated, and the spatial characteristics of a plurality of time-frequency points are clustered to determine a plurality of first sound wave signals of the clusters.

[0153] It should be noted that for each candidate sound source (preset sound source), the multipath components in the received signal are analyzed to construct a multipath delay fingerprint. If two or more signal components are separated from the mixed sound wave signal (i.e., the initial sound wave signal), the multipath delay fingerprint of each group of signals is determined, if the main direction of the two or more signal components is close, and the multipath delay fingerprints of the two or more signal components are different, it is determined that the two or more signal components originate from two different sound sources (i.e., physical sound sources).

[0154] It should be noted that the multiple signal classification algorithm or the rotation invariant subspace algorithm is used for sound source orientation to obtain angle resolution capability beyond the physical aperture limit.

[0155] 3. Determination process of sound source type

[0156] For example, as shown in Figure 6 The process of determining the sound source type of each first sound wave signal can include the following steps:

[0157] S601, for each first sound wave signal, the spatial characteristics and acoustic characteristics of the first sound wave signal are determined.

[0158] The spatial feature can include an azimuth angle and an elevation angle. The spatial feature can also be a direction vector.

[0159] Specifically, the cross-correlation function between different pairs of microphones is calculated.

[0160] For example, the cross-correlation function between different pairs of microphones can be calculated according to the following formula (8).

[0161] (8)

[0162] wherein, represents the cross-correlation function between the i-th microphone and the j-th microphone in the microphone array; represents the first sound signal collected by the i-th microphone; represents the first sound signal collected by the j-th microphone.

[0163] The time offset corresponding to the peak value of the cross-correlation function is determined as the time difference of arrival of the first sound signal.

[0164] For example, the time difference of arrival of the first sound signal can be calculated according to the following formula (9).

[0165] (9)

[0166] wherein, represents the time difference of arrival of the first sound signal.

[0167] Using the TDOA algorithm, the azimuth angle and the elevation angle of the first sound signal, i.e., the spatial feature of the first sound signal, are determined based on the time difference of arrival of different pairs of microphones.

[0168] The acoustic feature includes a time domain feature, a frequency domain feature, a cepstrum domain feature, and / or other stable and distinguishable acoustic features. The time domain feature includes a short-time energy and a zero-crossing rate. The short-time energy is used to represent the amplitude of the first sound signal and is used to distinguish different intensity sound sources. For example, a strong energy sound source (such as a truck) and a weak energy sound source (such as footstep sound).

[0169] The zero-crossing rate represents the number of times the signal crosses zero per unit time, and is used to determine the main frequency component of the sound signal (i.e., the first sound signal). For example, a high zero-crossing rate usually corresponds to high-frequency noise or clear speech. A low zero-crossing rate corresponds to low-frequency sound or murmur.

[0170] The frequency domain features include spectral centroid, spectral bandwidth and sub-band energy ratio. The spectral centroid represents the brightness of the sound signal, and is used to distinguish low-frequency centroid and high-frequency centroid. The high-frequency centroid is derived from, for example, metal friction and alarm sound. The low-frequency centroid is derived from, for example, engine roar and wind.

[0171] The spectral bandwidth describes the extension of the spectrum, and is used to distinguish narrow-band sound and wide-band sound. The narrow-band sound is, for example, whistle. The wide-band sound is, for example, white noise.

[0172] The sub-band energy ratio refers to the energy distribution of different specific frequency bands (such as 0-500Hz, 500-2kHz, 2k-8kHz), and is used to identify sound signals dominated by low-frequency energy, such as engine noise.

[0173] The cepstrum domain feature can be a mel-frequency cepstral coefficient (MFCC). The mel-frequency cepstral coefficient (MFCC) is a core feature of voiceprint recognition, and is used to simulate the hearing characteristics of human ears, compress the sound spectrum into a small number (such as 12-13) of key coefficients, and more finely represent the differences between sound sources. For example, to distinguish the footstep sounds of different pedestrians and the noises of different types of engines.

[0174] In addition, the time sequence dynamic mode of the mel-frequency cepstral coefficient is extracted, such as first-order and second-order difference, and a neural network is used to construct a dynamic relationship diagram of time-frequency-space features, i.e. a dynamic voiceprint atlas.

[0175] In S602, the sound source type of the sound source to which the first sound wave signal belongs is determined based on the spatial feature and the acoustic feature of the first sound wave signal.

[0176] In the embodiments of the present application, the sound source type can include a static sound source type, a dynamic sound source type, a self sound source type and a specified sound source type.

[0177] For the sound source belonging to the static sound source type (i.e. static sound source or general non-self sound source), the spatial feature and the acoustic feature are relatively stable, i.e. periodic or stationary. For example, the continuous humming sound of an air conditioner outdoor unit and the noise of a transformer.

[0178] For the sound source belonging to the dynamic sound source type (i.e. dynamic sound source), the spatial feature continuously changes (i.e. the spatial position continuously changes), and the acoustic feature can be stable or can change. For example, a passing car, whose engine noise spectrum is relatively stable, but the direction continuously changes.

[0179] For the sound source belonging to the self-sound source type (i.e., the self-sound source), the spatial feature is generally stable, the acoustic feature is known and fixed. It is usually the noise generated by the system itself (such as motor noise, fan sound). It can be filtered out. Because its position is known and fixed, it will not provide useful information for environment mapping, but will become interference. It can also be used for abnormal sound monitoring of the vehicle.

[0180] For the sound source belonging to the specified sound source type (i.e., the specified sound source), the spatial feature may be stable or variable, and the acoustic feature has a unique and identifiable time-frequency. For example, the siren sound of an ambulance has a specific periodic frequency change (pitch change); the car horn is a short-time burst of broadband signal; the brake sound is a high-frequency friction sound.

[0181] Based on this, when determining the sound source type of the first sound wave signal, the azimuth angle variance and the pitch angle variance of the first sound wave signal in the continuous time frame (multiple sampling points) are determined; the azimuth angle variance, the pitch angle variance and the acoustic feature are used to determine the sound source type of the sound source to which the first sound wave signal belongs.

[0182] Table 1 is the judgment basis of the sound source of different sound source types.

[0183]

[0184] For example, when the azimuth angle variance of the first sound wave signal is less than the preset azimuth angle variance threshold, and the pitch angle variance of the first sound wave signal is less than the preset pitch angle variance threshold, and the MFCC change rate of the first sound wave signal is less than the preset change rate threshold, the sound source type of the first sound wave signal is determined as the static sound source type, i.e., the static sound source.

[0185] When the azimuth angle variance of the first sound wave signal is less than the preset azimuth angle variance threshold, and the position of the first sound wave signal matches the preset position (the current position of the first vehicle), and the dynamic voiceprint spectrum of the first sound wave signal matches the preset voiceprint spectrum, the sound source type of the first sound wave signal is determined as the self-sound source type, i.e., the self-sound source.

[0186] When the azimuth angle variance of the first sound wave signal is greater than or equal to the preset azimuth angle variance threshold, or the pitch angle variance of the first sound wave signal is greater than or equal to the preset pitch angle variance threshold, the sound source type of the first sound wave signal is determined as the dynamic sound source type, i.e., the dynamic sound source.

[0187] When the spectral centroid of the first sound wave signal has a periodic change, or the dynamic voiceprint spectrum of the first sound wave signal matches the specified voiceprint spectrum, the sound source type of the first sound wave signal is determined as the specified sound source type, i.e., the specified sound source (or special sound source).

[0188] The preset pitch angle variance threshold is, for example, 5 degrees. The preset change rate threshold is 0.1.

[0189] 4. Static acoustic map construction process

[0190] For example, as shown in FIG. 7, the static acoustic map construction process can include the following steps: Figure 7

[0191] S701, feature extraction is performed on the second sound wave signal to determine an acoustic feature vector of the second sound wave signal.

[0192] Specifically, S7011, Fourier transform is performed on the second sound wave signal to obtain a time-frequency spectrum of the second sound wave signal.

[0193] For example, the time-frequency spectrum of the second sound wave signal can be determined according to the following formula (10).

[0194] (10)

[0195] wherein, represents the time-frequency spectrum of the second sound wave signal, represents the second sound wave signal collected by the i-th microphone. represents short-time Fourier transform.

[0196] S7012, based on the time-frequency spectrum of the second sound wave signal, the acoustic feature of the second sound wave signal is extracted to obtain the acoustic feature vector of the second sound wave signal.

[0197] The acoustic feature vector of the second sound wave signal includes at least one of a mel-frequency cepstral coefficient, a spectral centroid, a spectral bandwidth, and a sub-band energy ratio.

[0198] The mel-frequency cepstral coefficient usually takes 12-13 coefficients.

[0199] The spectral centroid can be represented as .

[0200] For example, the acoustic feature vector of the second sound wave signal can be represented as .

[0201] S702, based on the acoustic feature vector of each second sound wave signal, the multiple second sound wave signals are clustered and recognized to determine multiple static sound source groups.

[0202] wherein, the acoustic feature vectors of the multiple second sound wave signals can be represented as an acoustic feature vector set .

[0203] ​Specifically, distances between the acoustic feature vectors in the acoustic feature vector set are determined based on a preset neighborhood radius and a minimum point number, clustering is performed according to density distribution of the acoustic feature vectors in the preset neighborhood, and a clustering result is obtained.

[0204] For example, the plurality of second sound wave signals are clustered and identified according to the following formula (11).

[0205] (11)

[0206] wherein, represents the clustering result, represents an acoustic feature vector of the second sound wave signal, represents a neighborhood radius, represents a minimum point number, represents a density-based clustering algorithm.

[0207] For example, the neighborhood radius is 0.5, and the minimum point number is 3.

[0208] The clustering result can be represented as Each cluster represents a static sound source or a potential static sound source.

[0209] That is, the second sound wave signal is of a static sound source type. It can be understood that the plurality of second sound wave signals can include sound wave signals originating from the same sound source. In this regard, the plurality of second sound wave signals are clustered and identified, and a plurality of static sound source groups are determined according to the clustering result. One static sound source group can include a static sound source generating a corresponding sound wave signal, or can include a potential static sound source reflecting and refracting the sound wave signal emitted by the static sound source, i.e., a potential static sound source related to the static sound source. Therefore, the static sound source group can represent a group of acoustic reflecting units including the static sound source and the potential static sound source related to the static sound source, rather than a single sound source.

[0210] For example, an air conditioner outdoor unit (i.e., a static sound source) outside a building beside a road and a building wall (i.e., a potential static sound source) form a static sound source group.

[0211] S703, for each static sound source group, a plurality of sound source direction information groups of the static sound source group are determined according to the plurality of initial sound wave signal groups, and one group of sound source direction information is used to indicate a sound source direction of the static sound source and the potential static sound source in the static sound source group when the first vehicle is at any driving position.

[0212] Specifically, for each static sound source group, the sound source direction information of the static sound source and the potential static sound source in the static sound source group is determined by using the acoustic feature vectors of the static sound source and the potential static sound source in the static sound source group, that is, a set of sound source direction information of a static sound source group is obtained.

[0213] For example, after the static sound source group is determined, the second sound wave signals belonging to the static sound source and the second sound wave signals belonging to the potential static sound source are determined according to the acoustic feature vectors of the clustered second sound wave signals. The sound source direction information, such as the azimuth angle and the elevation angle, of the static sound source is determined by using the TDOA algorithm according to the time difference of arrival of the second sound wave signals belonging to the static sound source to different microphone pairs. The sound source direction information, such as the azimuth angle and the elevation angle, of the potential static sound source is determined by using the TDOA algorithm according to the time difference of arrival of the second sound wave signals belonging to the potential static sound source to different microphone pairs, and a set of sound source direction information of the static sound source group is obtained.

[0214] According to the above process, the sound source direction information of the static sound source and the potential static sound source in the static sound source group can be determined when the vehicle is in different driving positions.

[0215] S704, according to the multiple sets of sound source direction information of the static sound source group, the map point sequence of the static sound source group is determined, and the map point sequence is used to represent the relative position relationship between the static sound source group and the first vehicle during the driving of the first vehicle.

[0216] That is, the map point sequence is used to represent the relative position relationship between the static sound source in the static sound source group and the first vehicle, and the relative position relationship between the potential static sound source and the first vehicle during the driving of the first vehicle.

[0217] Specifically, according to the multiple initial sound wave signal groups collected by the vehicle in different driving positions, multiple sets of sound source direction information of the static sound source group relative to the vehicle are determined; based on the multiple sets of sound source direction information and the corresponding vehicle poses, a set of space rays is determined by triangulation method; the least square method is used to solve the best intersection point of the set of space rays, and the position estimate value of the intersection point is determined as the map point sequence of the static sound source group.

[0218] That is, for a static sound source group, according to the pose of the vehicle at different driving positions and the sound source direction corresponding to the static sound source, a set of space rays is determined by triangulation method, and the best intersection point of the set of space rays is solved by least square method, and the position estimate value of the intersection point is determined as the map point of the static sound source. And, according to the pose of the vehicle at different driving positions and the sound source direction corresponding to the potential static sound source, a set of space rays is determined by triangulation method, and the best intersection point of the set of space rays is solved by least square method, and the position estimate value of the intersection point is determined as the map point of the potential static sound source. The map point set composed of the map points of the static sound source and the map points of the potential static sound source is used as the map points of the static sound source group at the current driving position of the vehicle, and then the map point sequence of the static sound source group in the driving process of the vehicle is obtained.

[0219] The map point sequence of the static sound source group corresponds to the pose sequence of the first vehicle. For example, the pose sequence of the first vehicle can be represented as . Wherein, represents the pose of the first vehicle at t time. The map point sequence of the static sound source group can be represented as . Wherein, represents the position of the static sound source group when the pose of the first vehicle is . It should be noted that, represents the position of the static sound source and the potential static sound source in the static sound source group.

[0220] In some examples, after determining the map point sequence of the static sound source group according to the plurality of sound source direction information of the static sound source group, the method further comprises: obtaining the pose sequence of the first vehicle, the pose sequence being used to indicate the pose change of the first vehicle in the process of collecting the initial sound wave signal set; and optimizing the pose sequence and the map point sequence based on the error between the pose sequence and the map point sequence, to obtain the optimized pose sequence and the optimized map point sequence.

[0221] In some examples, the optimization of the pose sequence and the map point sequence based on the error between the pose sequence and the map point sequence, to obtain the optimized pose sequence and the optimized map point sequence, comprises: optimizing the pose sequence and the map point sequence based on a preset algorithm, with the goal of minimizing the error between the pose sequence and the map point sequence, to obtain the optimized pose sequence and the optimized map point sequence.

[0222] For example, the preset algorithm is Gauss-Newton method or Levenberg-Marquardt algorithm.

[0223] For example, the optimized pose sequence and the optimized map point sequence can be determined according to the following formula (12).

[0224] (12)

[0225] wherein, denotes the optimized pose sequence, denotes the optimized map point sequence, denotes the actual observation (e.g. azimuth or direction vector) of the map point at time t (i.e. the i-th static sound source group), denotes the current position (pose) and the map point predicted observation function. denotes the Mahalanobis distance, which is a distance metric that takes into account uncertainty, is the covariance matrix of the observation, which is used to represent the uncertainty of the error.

[0226] S705, constructing a static acoustic map of the environment where the first vehicle is located according to the map point sequence of the static sound source group and the static sound source feature of the static sound source group.

[0227] wherein, the static sound source feature of the static sound source group refers to a set of acoustic feature vectors of the static sound sources and potential static sound sources in the static sound source group.

[0228] The static acoustic map is used to indicate a plurality of static sound source groups in the environment where the first vehicle is located. The static acoustic map can structurally represent the spatial distribution and features of acoustic beacons (i.e. static sound source groups) in the environment where the first vehicle is located, i.e. a feature-based acoustic beacon database.

[0229] It should be noted that the static acoustic map can include sound source information corresponding to each static sound source group, i.e. sound source information of each static sound source and potential static sound source. The sound source information at least includes spatial position information and acoustic features (i.e. acoustic fingerprints) of the static sound source group. The spatial position information of the static sound source group is used to indicate the position of the static sound source group in the environment, such as coordinates, azimuth and pitch angle. The acoustic features (i.e. acoustic fingerprints) may, for example, be the acoustic feature vectors described above.

[0230] In addition, the sound source information can also include semantic labels. The semantic labels can represent the type of the static sound source group. It should be noted that after determining the static sound source group, the type of the static sound source group is determined according to the static sound source feature of the static sound source group, and the corresponding semantic label is generated according to the type of the static sound source group.

[0231] In some examples, a static acoustic map of the environment in which the first vehicle is located is constructed based on the pose sequence of the first vehicle, the map point sequence of the static sound source group, and the static sound source features of the static sound source group. That is, the static acoustic map can include sound source information corresponding to each static sound source group, i.e., the sound source information of each static sound source and potential static sound sources. The static acoustic map can also include the driving trajectory of the first vehicle, i.e., the pose sequence of the first vehicle.

[0232] In some embodiments, a static acoustic map of the environment in which the first vehicle is located is constructed based on the map point sequence of the static sound source group and the static sound source features of the static sound source group. This includes: constructing a static acoustic map of the environment in which the first vehicle is located based on the optimized pose sequence, the optimized map point sequence, and the static sound source features of the static sound source group. This can improve the construction accuracy of the static acoustic map.

[0233] In this embodiment, a group of static sound sources in the vehicle's environment is identified by clustering. A graph optimization framework is used to jointly optimize the vehicle's pose sequence and the map points of the static sound source group, which can improve the positioning accuracy of the static sound source group and thus improve the accuracy of the static acoustic map.

[0234] 5. Dynamic sound source tracking process

[0235] For example, such as Figure 8 As shown, the dynamic sound source tracking process may include the following steps:

[0236] S801, Obtain the orientation sequence of the dynamic sound source. The orientation sequence is used to characterize the relative positional relationship between the dynamic sound source and the first vehicle during the first vehicle's movement.

[0237] Specifically, the spectral characteristics of the third acoustic signal are extracted through time-frequency analysis, as shown in the following formula (13).

[0238] (13)

[0239] in, Let ζ represent the time spectrum, t be the center of the time frame, and w() be the window function used for local spectrum analysis. The extracted spectral features are used for classification. This represents the preprocessed sound wave signal collected by the i-th microphone.

[0240] The spectral features of the third acoustic signal are extracted, and a machine learning classifier is used to classify these features and identify multiple dynamic sound sources. The machine learning classifier can be a support vector machine, convolutional neural network, or similar technology.

[0241] In some examples, after classifying the spectral features of the third sound wave signal, a semantic label can be added for each dynamic sound source.

[0242] S802, according to the orientation sequence of the dynamic sound source, the motion trajectory of the dynamic sound source is determined.

[0243] Specifically, a tracker based on Kalman filtering or particle filtering is used, and the state vector of the tracker. The state vector represents the position and moving speed of the dynamic sound source. The state vector can be represented as .

[0244] The tracking process of the dynamic sound source includes a prediction step and an update step. Wherein,

[0245] Taking Kalman filtering as an example, the prediction step: the execution state prediction is performed according to the following formula (14).

[0246] (14)

[0247] Wherein, represents the optimal state estimation at the k-1 time, represents the state transition matrix (prediction model) at the current time (i.e. the k time), represents the state of the dynamic sound source at the k time predicted by using the state at the k-1 time and the motion model.

[0248] The covariance prediction is performed according to the following formula (15).

[0249] (15)

[0250] Wherein, represents the uncertainty (covariance prediction) of the state estimation at the k-1 time, represents the state transition matrix (prediction model) at the current time (i.e. the k time), represents the uncertainty of the predicted state, represents the process noise.

[0251] The update step: the Kalman gain is calculated according to the following formula (16).

[0252] (16)

[0253] Wherein, represents the Kalman gain, represents the uncertainty (covariance prediction) of the state estimation at the k-1 time, is the observation matrix, represents the covariance matrix of the observation noise.

[0254] The state update is performed according to the following formula (17).

[0255] (17)

[0256] wherein, denotes the optimal state estimation at time k, denotes the Kalman gain, denotes the bearing sequence of the dynamic sound source, is the observation matrix, denotes the optimal state estimation at time k-1.

[0257] The covariance is updated according to the following formula (18).

[0258] (18)

[0259] wherein, denotes the uncertainty of the updated state estimation, denotes the Kalman gain, is the observation matrix.

[0260] In the embodiments of the present application, the Kalman filter or the particle filter is used to predict the trajectory of the dynamic sound source (such as a vehicle), so that the motion trajectory of the dynamic sound source can be predicted more accurately.

[0261] In some embodiments, when the observation of the dynamic sound source is incomplete, prior knowledge is used for reasoning. It is assumed that a target vehicle to be tracked can only estimate the bearing angle θ at a certain time due to occlusion, but cannot accurately obtain the distance r or completely determine the identity. However, it is known that the map information is, for example, that the vehicle is driving on a lane with a direction of . Optionally, the particle filter or the extended Kalman filter is used to fuse the observation and the prior (other filters can also be used, which are only examples here). First, the state vector (position, velocity) of the vehicle is obtained, and the constant speed model (or other motion model) is used to predict the state. The prediction step can refer to the following formula (19):

[0262] (19)

[0263] wherein, is the state transition matrix. It should be noted that has the same mathematical meaning as in the formula (14) above, and needs to be distinguished in actual application.

[0264] The update step is used to fuse the observation and the prior. According to the bearing angle information provided by the incomplete observation, and the prior information (the lane direction can be converted into a soft constraint on the target motion direction), the fusion likelihood function can be constructed according to the following formula (20), for example:

[0265] (20)

[0266] where, is the likelihood of bearing observation, optionally, following a Gaussian distribution , is the prior likelihood, which can be constructed by itself, for example, if the target vehicle is driving on the lane, it can be set as: , indicating that the closer the target motion direction is to the lane direction, the higher the prior probability. Other constructions are the same.

[0267] Then, the state can be updated according to Bayes' theorem, see formula (21) below:

[0268] (21)

[0269] In the above manner, even if the observation is incomplete, the prior knowledge (lane direction) can help to narrow down the possible state space of the target, and provide more accurate position and trajectory prediction, especially in non-line-of-sight scenarios.

[0270] It should be noted that during the movement of the vehicle, the clusters of sound sources that cannot be distinguished at the moment (dynamic sound sources) are jointly tracked. With the change of observation geometry, the separation of trajectories in the state space (such as the bearing-time graph) is used to decouple the sound source cluster and initialize two independent sound source targets (dynamic sound sources) for tracking.

[0271] 6. Conflict resolution process of dynamic and static information

[0272] In some embodiments, before fusing the static acoustic map and the motion trajectory of each dynamic sound source to obtain the map data of the environment in which the first vehicle is located, the static sound source group and the dynamic sound source are updated.

[0273] For example, as shown in Figure 9 , the process of updating the static sound source group and the dynamic sound source can include the following steps:

[0274] S901, obtaining a sound source observation set collected by a microphone array.

[0275] Specifically, the sound source observation set collected by the microphone array can be represented as , wherein each observation contains attributes such as bearing and acoustic characteristics.

[0276] S902, generating a set of mutually exclusive tracking hypotheses according to the sound source observation set.

[0277] The tracking hypotheses include: dynamic target hypothesis H1 (associated with an existing trajectory or a new trajectory in the dynamic trajectory set Y k ) and static target hypothesis H2 (associated with an existing map point or a new landmark in the static acoustic map L k ). The prior probability of the hypotheses is modulated by the semantic information of the static acoustic map L k .

[0278] S903, iteratively update the posterior probability of the tracking hypotheses with the set of sound source observations.

[0279] With the new observation data , the posterior probability of each hypothesis is iteratively updated by Bayes' theorem: .

[0280] where, is the posterior probability, representing the probability of hypothesis H being true after considering all the current observations , the current static map and the current dynamic trajectory set . is the likelihood function, measuring how likely it is to observe the current data if hypothesis H is true.

[0281] Each hypothesis competitively explains the observation data in a unified probability framework, and its probability evolves over time series according to the observation consistency.

[0282] S904, according to the updated posterior probability, output the updated map point sequence of the static sound source group (i.e. the updated static acoustic map) and the updated motion trajectory of the dynamic sound source (i.e. the new dynamic trajectory set).

[0283] According to the updated posterior probability, select the optimal hypothesis (such as using the MAP criterion). The observation confirmed as H2 hypothesis is fused to generate the updated static acoustic map L k+1 . The observation confirmed as H1 hypothesis is used for tracking to generate the updated dynamic trajectory set Y k+1 .

[0284] For competing hypotheses with close probabilities, the system maintains their parallel evolution until their probabilities significantly separate in subsequent observations.

[0285] This mechanism enables the system to handle quasi-static targets, whose probability converges to H2 when stationary and quickly switches to H1 when moving.

[0286] Subsequently, when fusing the static acoustic map with the motion trajectories of each dynamic sound source to obtain map data of the environment where the first vehicle is located, a static acoustic map of the environment where the first vehicle is located is constructed based on the updated map point sequence of the static sound source group and the static sound source characteristics of the static sound source group; the static acoustic map with the updated motion trajectories of each dynamic sound source is then fused to obtain map data of the environment where the first vehicle is located.

[0287] The embodiments of this application can effectively avoid map pollution or tracking failure caused by misjudgment of sound sources, ensure reliable separation and fusion of static and dynamic sound sources in complex acoustic environments, and improve the accuracy of map data.

[0288] The following is a specific example illustrating the map generation method provided in this application.

[0289] Figure 10 This is a flowchart illustrating a map generation method provided in an embodiment of this application. Figure 10 As shown, the map generation method includes the following steps:

[0290] S1001, acquire the set of mixed sound wave signals collected by the microphone array at different driving positions (i.e., the initial set of sound wave signals mentioned above).

[0291] S1002, determine whether the average spectral entropy of the mixed acoustic signal is less than the preset spectral entropy threshold. If yes, execute S1004; otherwise, execute S1003.

[0292] S1003 is set to dense acoustic environment mode, maintaining passive reception mode.

[0293] S1004 indicates a sparse acoustic environment, triggering the active sound generation module to operate.

[0294] S1005, Obtain the echo signal of the sound wave signal emitted by the active sound generation module received by the microphone array.

[0295] S1006, Process the mixed sound wave signal set, identify multiple valid sound wave signals, extract features from the valid sound wave signals, and determine the type of sound source to which the valid sound wave signals belong.

[0296] S1007, Determine that the sound source to which the valid sound wave signal belongs is a static sound source.

[0297] S1008 identifies the effective sound wave signal, determines multiple static sound source groups in the environment where the first vehicle is located, and constructs a static acoustic map of the environment where the first vehicle is located based on the characteristic information of each static sound source group.

[0298] S1009, determine that the sound source to which the valid sound wave signal belongs is a dynamic sound source.

[0299] S1010, positioning each valid sound wave signal to determine the motion trajectory of the dynamic sound source corresponding to the valid sound wave signal.

[0300] S1011, conflict resolution for the static sound source group and the dynamic sound source.

[0301] Specifically, the specific implementation of S1011 can refer to the specific implementation of S901 to S904 described above, which will not be repeated here.

[0302] S1012, fusing the static acoustic map and the motion trajectory of each dynamic sound source to obtain the map data of the environment where the first vehicle is located.

[0303] In the embodiments of the application, a passive sensing mechanism is adopted, and non-line-of-sight detection is supported; by introducing an acoustic scene modeling and target tracking engine, the system can convert the dynamic sound signals in the environment from interference terms into effective sensing information sources, thereby simultaneously realizing reconstruction of the static structure of the environment and tracking of dynamic targets, and achieving comprehensive environmental perception.

[0304] The embodiments of the application can realize collaborative perception and modeling of static environment structure and dynamic targets, the core of which is to use the same set of acoustic signal streams to serve static map construction and dynamic target tracking, and to ensure the consistency of the two types of information through a conflict resolution mechanism, thereby constructing an integrated dynamic and static environment cognition model.

[0305] Specifically, by analyzing the acoustic fingerprint characteristics of the sound source and the multipath effect (reflection path difference), the system can not only identify the type of the sound source, but also infer the existence and position of the reflection surface, thereby realizing identification of dynamic targets and modeling of static obstacles at the same time.

[0306] The embodiments of the application also provide a vehicle control method. As shown in the figure, the vehicle control method comprises the following steps: Figure 11

[0307] S1101, obtaining map data of an environment where a first vehicle is located.

[0308] S1102, generating a first auxiliary driving instruction according to the map data.

[0309] S1103, executing the first auxiliary driving instruction to control the first vehicle to travel.

[0310] In some embodiments, the vehicle control method further comprises: after the first vehicle and the second vehicle establish a communication connection, sending the map data to the second vehicle, so that the second vehicle generates a second auxiliary driving instruction according to the map data, and travels according to the second auxiliary driving instruction.

[0311] ​For example, the map data can be converted into a compact data structure. Each map point in the map point sequence of the static sound source group contains its three-dimensional coordinates and its acoustic feature descriptor (such as the mean and variance of the mel-frequency cepstral coefficient vector), so as to facilitate transmission and sharing.

[0312] In some embodiments, after receiving the map data sent by other vehicles, the first vehicle compares and fuses the generated map data and the map data sent by other vehicles to obtain updated map data.

[0313] That is, the respective constructed map data (i.e., local sound field map) of different vehicles can be exchanged, and after receiving the local sound field map of other vehicles, the generated local sound field map and the local sound field map of other vehicles are fused.

[0314] For example, the map point sequence of the static sound source group in the local sound field map of the first vehicle is feature-matched with the map point sequence of the static sound source group in the local sound field map of other vehicles to determine the common static sound source group (i.e., common landmark). After feature matching, the coordinate transformation relationship between the local sound field map of the first vehicle and the local sound field map of other vehicles is determined. Based on the coordinate transformation relationship, data fusion is performed on the feature-matched static sound source group, and the map point of the static sound source group that exists in the local sound field map of other vehicles but is not contained in the local sound field map of the first vehicle is added to the local sound field map of the first vehicle.

[0315] In this way, through the sharing mechanism, a vehicle newly entering the environment can quickly obtain the prior map, thereby realizing "instant positioning" and effectively avoiding the cold start bottleneck. At the same time, the crowdsourcing data contribution mode also significantly improves the sensing of environmental changes and the agility of map updating of the system.

[0316] In addition, when transmitting map data between vehicles, any combination of the following techniques can be used:

[0317] (1) Forward error correction coding: redundant information is added in the data packet, so that the receiving end can correct errors by itself when encountering a certain degree of error code or packet loss, without the need for retransmission. Specifically, at the data sending end, the sensor data (such as time difference measurement value, feature vector) is forward error correction coded. Common codecs include Reed-Solomon code or convolution code. At the same time, at the data receiving end, forward error correction decoding is performed to attempt to recover the original data.

[0318] Taking a systematic code as an example, the original information bit is u, and the generation matrix is G. Then the coded code word is When the receiving end receives an error code word , the decoding algorithm finds the expected code word closest to the coded code word, and recovers the original information bit u.

[0319] (2) Automatic repeat request mechanism: The data is reliably arrived by the acknowledgement of the receiving end and the retransmission of the sending end. Specifically, the sending end sets a sequence number and a timer for each data packet. The receiving end sends back an acknowledgement message after successfully receiving the data packet. If the sending end does not receive the acknowledgement message before the timer expires, it considers that the data packet is lost and will retransmit it. Because the automatic repeat request increases the delay, this mechanism is recommended to be applied to the transmission of control instructions or key map data with slightly lower real-time requirement but extremely high reliability requirement.

[0320] (3) Adaptive modulation and coding: According to the real-time change of the wireless channel quality, the sending strategy is dynamically adjusted to improve the rate when the channel condition is good and to improve the reliability when the channel condition is poor. Specifically, the sending end or the receiving end estimates the channel state information such as the signal-to-noise ratio. According to the preset signal-to-noise ratio-modulation coding scheme threshold table, the most suitable modulation coding scheme is selected. For example, high-order modulation (such as 64-QAM) and high code rate are selected when the signal-to-noise ratio is high, and low-order modulation (such as binary phase shift keying) and low code rate (more redundancy) are selected when the signal-to-noise ratio is low.

[0321] Optionally, in the actual system, a hybrid automatic repeat request, that is, the combination of forward error correction and automatic repeat request, can be used. First, the forward error correction is used to correct errors, and if the errors exceed the forward error correction capability, retransmission is requested.

[0322] The embodiment of the present application further provides a map generation device. As shown in FIG. 2, the map generation device comprises a map generation unit 21, a map storage unit 22, a map transmission unit 23, a map receiving unit 24, a map display unit 25 and a map update unit 26. Figure 12As shown, the map generation apparatus 1200 includes a signal acquisition module 1201, a determination module 1202, a static map construction module 1203, a dynamic trajectory generation module 1204, and a fusion module 1205. The signal acquisition module 1201 is configured to acquire a set of initial sound wave signals collected by the microphone array at different driving positions during driving of the first vehicle. The determination module 1202 is configured to process the set of initial sound wave signals to determine a plurality of first sound wave signals, the plurality of first sound wave signals including a plurality of second sound wave signals and a plurality of third sound wave signals, the second sound wave signals belonging to static sound sources of a static sound source type, and the third sound wave signals belonging to dynamic sound sources of a dynamic sound source type. The static map construction module 1203 is configured to identify the plurality of second sound wave signals, determine a plurality of static sound source groups in an environment in which the first vehicle is located, and construct a static acoustic map of the environment in which the first vehicle is located based on sound source information of each static sound source group. The dynamic trajectory generation module 1204 is configured to locate each third sound wave signal to obtain a dynamic sound source to which each third sound wave signal belongs, and determine a motion trajectory of each dynamic sound source. The fusion module is configured to fuse the static acoustic map and the motion trajectories of the dynamic sound sources to obtain map data of the environment in which the first vehicle is located. The fusion module 1205 is configured to fuse the static acoustic map and the motion trajectories of the dynamic sound sources to obtain map data of the environment in which the first vehicle is located.

[0323] In some embodiments, the static map construction module 1203 is specifically configured to extract features of the second sound wave signals to determine acoustic feature vectors of the second sound wave signals, and perform cluster identification on the plurality of second sound wave signals based on the acoustic feature vectors of the second sound wave signals to determine the plurality of static sound source groups.

[0324] In some embodiments, the acoustic feature vector of the second sound wave signal includes at least one of a mel-frequency cepstral coefficient, a spectral centroid, a spectral bandwidth, and a sub-band energy ratio.

[0325] In some embodiments, the set of initial sound wave signals includes a plurality of groups of initial sound wave signals collected at different driving positions. The static map construction module 1203 is specifically configured to, for each static sound source group, determine a plurality of sets of sound source direction information of the static sound source group from the plurality of groups of initial sound wave signals, one set of sound source direction information being used to indicate sound source directions of static sound sources and potential static sound sources in the static sound source group when the first vehicle is at any driving position, determine a map point sequence of the static sound source group based on the plurality of sets of sound source direction information of the static sound source group, the map point sequence being used to represent a relative positional relationship between the static sound source group and the first vehicle during driving of the first vehicle, and construct the static acoustic map of the environment in which the first vehicle is located based on the map point sequence of the static sound source group and static sound source features of the static sound source group.

[0326] In some embodiments, the map generation apparatus 1200 further comprises a pose obtaining module. The pose obtaining module is configured to obtain a pose sequence of the first vehicle, the pose sequence being used to indicate a pose change of the first vehicle in a process of collecting the initial sound wave signal set. The static map construction module 1203 is further configured to optimize the pose sequence and the map point sequence based on an error between the pose sequence and the map point sequence, to obtain an optimized pose sequence and an optimized map point sequence; and construct a static acoustic map of an environment in which the first vehicle is located according to the optimized pose sequence, the optimized map point sequence and the static sound source features of the static sound source group.

[0327] In some embodiments, the static map construction module 1203 is specifically configured to optimize the pose sequence and the map point sequence based on a preset algorithm, with a target of minimizing the error between the pose sequence and the map point sequence, to obtain the optimized pose sequence and the optimized map point sequence.

[0328] In some embodiments, the map generation apparatus 1200 further comprises a conflict resolution module configured to obtain a sound source observation set collected by the microphone array; generate a set of mutually exclusive tracking hypotheses according to the sound source observation set, the tracking hypotheses including a dynamic target hypothesis associated with a motion trajectory of a dynamic sound source and a static target hypothesis associated with the map point sequence of the static sound source group; iteratively update a posterior probability of the tracking hypotheses by using the sound source observation set; and output an updated motion trajectory of the dynamic sound source and an updated map point sequence of the static sound source group according to the updated posterior probability. The fusion module 1205 is specifically configured to construct a static acoustic map of an environment in which the first vehicle is located according to the updated map point sequence of the static sound source group and the static sound source features of the static sound source group; and fuse the static acoustic map and the updated motion trajectory of each dynamic sound source to obtain map data of the environment in which the first vehicle is located.

[0329] In some embodiments, the dynamic trajectory generation module 1204 is specifically configured to obtain a bearing sequence of the dynamic sound source, the bearing sequence being used to represent a relative positional relationship between the dynamic sound source and the first vehicle during a driving process of the first vehicle; and determine the motion trajectory of the dynamic sound source according to the bearing sequence of the dynamic sound source.

[0330] In some embodiments, the initial sound wave signal set comprises a plurality of groups of initial sound wave signals, one group of initial sound wave signals comprising initial sound wave signals collected by each microphone in the microphone array, and one initial sound wave signal collected by one microphone being a mixed sound wave signal composed of a plurality of sound wave signals from different sound sources in an environment in which the first vehicle is located. The determination module 1202 is specifically configured to perform sound source positioning and separation on each group of initial sound wave signals to determine a plurality of first sound wave signals; and determine a sound source type of a sound source to which each first sound wave signal belongs.

[0331] In some embodiments, the map generation apparatus 1200 further includes a preprocessing module configured to perform filtering processing on the initial sound wave signals to obtain filtered initial sound wave signals, perform multipath suppression processing on the filtered initial sound wave signals to obtain multipath-suppressed initial sound wave signals, and perform signal enhancement processing on the multipath-suppressed initial sound wave signals to obtain enhanced initial sound wave signals.

[0332] In some embodiments, the plurality of first sound wave signals includes a fourth sound wave signal of a self sound source type; and the map generation apparatus 1200 further includes a removal module configured to remove the fourth sound wave signal from the plurality of first sound wave signals.

[0333] In some embodiments, the plurality of first sound wave signals includes a fifth sound wave signal of a specified sound source type; and the map generation apparatus 1200 further includes an addition module configured to add a specified label to a static sound source group in the map data when the type of the static sound source group is a specified type, and add the specified label to a dynamic sound source in the map data when the type of the dynamic sound source is the specified type.

[0334] The embodiments of the present application further provide a vehicle control apparatus. As shown in Figure 13 the vehicle control apparatus 1300 includes a data acquisition module 1301, an instruction generation module 1302, and a control module 1303. The data acquisition module 1301 is configured to acquire map data of an environment in which a first vehicle is located. The instruction generation module 1302 is configured to acquire the map data of the environment in which the first vehicle is located. The control module 1303 is configured to execute a first auxiliary driving instruction to control the first vehicle to travel.

[0335] In some embodiments, the vehicle control apparatus 1300 further includes a sending module configured to send the map data to a second vehicle after the first vehicle and the second vehicle establish a communication connection, so that the second vehicle generates a second auxiliary driving instruction according to the map data and travels according to the second auxiliary driving instruction.

[0336] The vehicle provided by the embodiments can execute the method provided by the above-mentioned method embodiments, and has similar implementation principles and technical effects, which will not be described here in detail.

[0337] Figure 14 A structural schematic diagram of a vehicle provided by the present application is shown in Figure 14 As shown in the figure, the vehicle 140 provided by the embodiments includes at least one processor 1401 and a memory 1402. Optionally, the vehicle 140 further includes a communication component 1403. The processor 1401, the memory 1402, and the communication component 1403 are connected through a bus 1404.

[0338] In the implementation process, the at least one processor 1401 executes computer execution instructions stored in the memory 1402, so that the at least one processor 1401 executes the above-mentioned method.

[0339] The specific implementation process of the processor 1401 can refer to the method embodiments described above, which have similar implementation principles and technical effects, and details are not described here.

[0340] In the above embodiments, it should be understood that the processor can be a central processing unit (English: Central Processing Unit, CPU for short), and can also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, DSP for short), application specific integrated circuits (English: Application Specific Integrated Circuit, ASIC for short), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as hardware processor execution or combined with hardware and software modules in the processor for execution.

[0341] The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), such as at least one disk memory.

[0342] The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, the bus in the drawings of the present application does not limit to only one bus or one type of bus.

[0343] The present application also provides a computer program product, comprising a computer program, which is executed by a processor to implement the above-mentioned method.

[0344] The present application also provides a computer readable storage medium, which stores computer execution instructions, and when the processor executes the computer execution instructions, the above-mentioned method is implemented.

[0345] The above-mentioned readable storage medium can be realized by any type of volatile or nonvolatile storage devices or their combinations, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0346] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0347] The division of units is only a logical functional division, and in actual implementation, there can be another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0348] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0349] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0350] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0351] It can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0352] Finally, it should be noted that: those skilled in the art will easily think of other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application that follow the general principles of the present application and include common knowledge or conventional technical means in the art that are not disclosed in the present application, and is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is only limited by the appended claims.

Claims

1. A map generation method, characterized in that, Applied to a first vehicle, wherein a microphone array is provided on the first vehicle, the method includes: During the driving process of the first vehicle, the initial sound wave signal set collected by the microphone array at different driving positions is acquired, and the initial sound wave signal set includes multiple initial sound wave signal groups collected at different driving positions. The initial set of sound wave signals is processed to determine multiple first sound wave signals, which include multiple second sound wave signals and multiple third sound wave signals. The sound source to which the second sound wave signal belongs is a static sound source type, and the sound source to which the third sound signal belongs is a dynamic sound source type. The plurality of second acoustic signals are identified to determine a plurality of static sound source groups in the environment in which the first vehicle is located. For each static sound source group, multiple sets of sound source direction information are determined based on the plurality of initial acoustic signal groups. Based on the multiple sets of sound source direction information, a map point sequence of the static sound source group is determined. Based on the map point sequence of the static sound source group and the static sound source characteristics of the static sound source group, a static acoustic map of the environment in which the first vehicle is located is constructed. In this map point sequence, a set of sound source direction information is used to indicate the sound source direction of the static sound sources and potential static sound sources in the static sound source group when the first vehicle is in any driving position. The map point sequence is used to characterize the relative positional relationship between the static sound source group and the first vehicle during the driving process of the first vehicle. Each of the third sound wave signals is located to obtain the dynamic sound source to which each of the third sound wave signals belongs, and the motion trajectory of each dynamic sound source is determined. The static acoustic map and the motion trajectories of each dynamic sound source are fused to obtain map data of the environment in which the first vehicle is located.

2. The method according to claim 1, characterized in that, The step of identifying the plurality of second acoustic signals to determine the plurality of static sound source groups in the environment where the first vehicle is located includes: Feature extraction is performed on the second acoustic signal to determine the acoustic feature vector of the second acoustic signal; Based on the acoustic feature vectors of each of the second acoustic signals, the multiple second acoustic signals are clustered and identified to determine multiple groups of static sound sources.

3. The method according to claim 2, characterized in that, The acoustic feature vector of the second acoustic signal includes at least one of the following: Mel frequency cepstral coefficient, spectral centroid, spectral bandwidth, and subband energy ratio.

4. The method according to claim 1, characterized in that, After determining the map point sequence of the static sound source group based on multiple sets of sound source direction information, the method further includes: The pose sequence of the first vehicle is obtained, and the pose sequence is used to indicate the pose changes of the first vehicle during the acquisition of the initial set of acoustic signals. Based on the error between the pose sequence and the map point sequence, the pose sequence and the map point sequence are optimized to obtain optimized pose sequence and optimized map point sequence. The step of constructing a static acoustic map of the environment in which the first vehicle is located based on the map point sequence of the static sound source group and the static sound source characteristics of the static sound source group includes: Based on the optimized pose sequence, the optimized map point sequence, and the static sound source features of the static sound source group, a static acoustic map of the environment in which the first vehicle is located is constructed.

5. The method according to claim 4, characterized in that, The step of optimizing the pose sequence and the map point sequence based on the error between them to obtain optimized pose sequences and optimized map point sequences includes: With the goal of minimizing the error between the pose sequence and the map point sequence, the pose sequence and the map point sequence are optimized based on a preset algorithm to obtain the optimized pose sequence and the optimized map point sequence.

6. The method according to claim 1, characterized in that, Before fusing the static acoustic map and the motion trajectories of each dynamic sound source to obtain map data of the environment in which the first vehicle is located, the method further includes: Obtain the set of sound source observations collected by the microphone array; Based on the sound source observation set, a set of mutually exclusive tracking hypotheses is generated, the tracking hypotheses including dynamic target hypotheses associated with the motion trajectory of the dynamic sound source and static target hypotheses associated with the map point sequence of the static sound source group; The posterior probability of the tracking hypothesis is iteratively updated using the aforementioned sound source observation set; Based on the updated posterior probabilities, output the updated map point sequence of the static sound source group and the updated motion trajectory update of the dynamic sound source; The process of fusing the static acoustic map with the motion trajectories of each dynamic sound source to obtain map data of the environment in which the first vehicle is located includes: The static acoustic map of the environment where the first vehicle is located is constructed based on the updated map point sequence of the static sound source group and the static sound source characteristics of the static sound source group. The static acoustic map and the updated motion trajectories of each dynamic sound source are fused to obtain map data of the environment in which the first vehicle is located.

7. The method according to claim 1, characterized in that, Determining the motion trajectory of each dynamic sound source includes: Obtain the orientation sequence of the dynamic sound source, the orientation sequence being used to characterize the relative positional relationship between the dynamic sound source and the first vehicle during the first vehicle's movement; The motion trajectory of the dynamic sound source is determined based on the directional sequence of the dynamic sound source.

8. The method according to claim 1, characterized in that, The initial sound wave signal set includes multiple sets of initial sound wave signals. Each set of initial sound wave signals includes initial sound wave signals collected by each microphone in the microphone array. The initial sound wave signal collected by one microphone is a mixed sound wave signal composed of multiple sound wave signals from different sound sources in the environment where the first vehicle is located. The processing of the initial sound wave signal set to determine multiple first sound wave signals includes: For each group of initial acoustic signals, sound source localization and separation are performed to determine the plurality of first acoustic signals; Determine the sound source type of each first sound wave signal.

9. The method according to claim 8, characterized in that, The method further includes: The initial acoustic signal is filtered to obtain a filtered initial acoustic signal. The filtered initial acoustic signal is subjected to multipath suppression processing to obtain a multipath suppressed initial acoustic signal. The initial acoustic signal after multipath suppression is subjected to signal enhancement processing to obtain an enhanced initial acoustic signal.

10. The method according to claim 8, characterized in that, The plurality of first acoustic signals include a fourth acoustic signal whose sound source type is a self-sound source type; After processing the initial set of acoustic signals to determine multiple first acoustic signals, the method further includes: Remove the fourth acoustic signal from the plurality of first acoustic signals.

11. The method according to claim 8, characterized in that, The plurality of first acoustic signals includes a fifth acoustic signal whose sound source type is a specified sound source type; the method further includes: If the type of the static sound source group is a specified type, add a specified label to the static sound source group in the map data; When the type of the dynamic sound source is the specified type, the specified label is added to the dynamic sound source in the map data.

12. A vehicle control method, characterized in that, Applied to a first vehicle, the method includes: Obtain map data of the environment in which the first vehicle is located, wherein the map data is generated according to the map generation method as described in any one of claims 1-11; Based on the map data, a first assisted driving command is generated; The first assisted driving command is executed to control the movement of the first vehicle.

13. The method according to claim 12, characterized in that, The method further includes: After the first vehicle establishes a communication connection with the second vehicle, the map data is sent to the second vehicle so that the second vehicle can generate a second assisted driving command based on the map data and drive according to the second assisted driving command.

14. A map generation apparatus, characterized in that, The device is configured in a first vehicle, on which a microphone array is provided, and the device includes: The signal acquisition module is used to acquire, during the driving of the first vehicle, an initial set of sound wave signals collected by the microphone array at different driving positions, wherein the initial set of sound wave signals includes multiple groups of initial sound wave signals collected at different driving positions. The determination module is used to process the initial set of sound wave signals and determine multiple first sound wave signals, wherein the multiple first sound wave signals include multiple second sound wave signals and multiple third sound wave signals, wherein the type of sound source to which the second sound wave signal belongs is a static sound source type, and the type of sound source to which the third sound signal belongs is a dynamic sound source type; A static map construction module is used to identify the plurality of second acoustic signals, determine a plurality of static sound source groups in the environment where the first vehicle is located, and for each static sound source group, determine multiple sets of sound source direction information of the static sound source group based on the plurality of initial acoustic signal groups, determine the map point sequence of the static sound source group based on the multiple sets of sound source direction information of the static sound source group, and construct a static acoustic map of the environment where the first vehicle is located based on the map point sequence of the static sound source group and the static sound source characteristics of the static sound source group. A set of sound source direction information is used to indicate the sound source direction of static sound sources and potential static sound sources in the static sound source group when the first vehicle is at any driving position, and the map point sequence is used to characterize the relative positional relationship between the static sound source group and the first vehicle during the first vehicle's movement. The dynamic trajectory generation module is used to locate each of the third sound wave signals, obtain the dynamic sound source to which each of the third sound wave signals belongs, and determine the motion trajectory of each dynamic sound source. The fusion module is used to fuse the static acoustic map and the motion trajectories of each dynamic sound source to obtain map data of the environment in which the first vehicle is located.

15. A vehicle control device, characterized in that, The device is configured in a first vehicle and includes: The data acquisition module is used to acquire map data of the environment where the first vehicle is located, wherein the map data is generated according to the map generation device as described in claim 14; The instruction generation module is used to generate a first assisted driving instruction based on the map data; The control module is used to execute the first assisted driving command to control the movement of the first vehicle.

16. A vehicle, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as claimed in any one of claims 1-11, or causing the processor to perform the method as claimed in claim 12 or 13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-11, or to implement the method as described in claim 12 or 13.

Images