Target detection method and device, vehicle and storage medium

Abstract

The invention provides a target detection method and device, a vehicle and a storage medium, and relates to the field of automatic driving, and the method comprises the steps: collecting a sound wave signal in an environment where the vehicle is located through a microphone array of the vehicle; identifying a target sound source in the environment according to the sound wave signal; measuring the time difference of the sub sound wave signals of the target sound source in the sound wave signals reaching any two microphones in the microphone array; and positioning the current position of the target sound source according to the time difference and the installation positions of any two microphones. Therefore, the target sound source (such as a moving object) in the environment where the vehicle is located and the position where the target sound source is located are identified by utilizing the penetrability of sound waves and the sound wave signals captured by the microphone array installed on the vehicle, the accuracy and comprehensiveness of target sound source identification can be improved, the situation that the shielded sound source or the moving object is missed to be identified is avoided, and the identification efficiency is improved. And therefore, the safety of vehicle driving can be improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a target detection method, device, vehicle and storage medium. Background Art

[0002] In related technologies, obstacles in the vehicle's environment can be detected based on the vehicle's sensing equipment (such as on-board cameras, ultrasonic radars, lidars, millimeter-wave radars, etc.) to obtain information related to the obstacles (such as the category and location of the obstacles).

[0003] However, when a vehicle is traveling in an environment with many obstructions (such as an underground parking lot), the sensing device cannot effectively detect obstacles or moving objects (such as moving vehicles, pedestrians, etc.) blocked by obstructions (such as walls, etc.). For example, the sensing device cannot effectively detect the "ghost poking its head" scene caused by the obstruction. When the moving object is detected by the sensing device, braking may be performed too late, resulting in a vehicle collision accident, which seriously reduces the safety of vehicle driving. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the present application proposes a target detection method, device, vehicle and storage medium to improve the accuracy and comprehensiveness of target sound source (such as moving objects) identification, avoid missing the identification of obscured sound sources or moving objects, and cause vehicle collision accidents, thereby improving the safety of vehicle driving.

[0006] In one aspect, an embodiment of the present application provides a target detection method, including:

[0007] Collecting sound wave signals in the environment where the vehicle is located through a microphone array of the vehicle;

[0008] identifying a target sound source in the environment according to the sound wave signal;

[0009] Measuring the time difference between the sub-sound wave signal of the target sound source in the sound wave signal and the arrival of any two microphones in the microphone array;

[0010] The current position of the target sound source is located according to the time difference and the installation positions of any two microphones.

[0011] Another aspect of the present application provides an object detection device, including:

[0012] A collection module, used to collect sound wave signals in the environment where the vehicle is located through a microphone array of the vehicle;

[0013] an identification module, used to identify a target sound source in the environment according to the sound wave signal;

[0014] A measuring module, used for measuring the time difference between the sub-sound wave signal of the target sound source in the sound wave signal and the arrival of any two microphones in the microphone array;

[0015] A positioning module is used to locate the current position of the target sound source according to the time difference and the installation positions of any two microphones.

[0016] Another aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the target detection method as described in the aforementioned aspect is implemented.

[0017] Another aspect of the present application provides a vehicle, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the target detection method as described in the aforementioned aspect.

[0018] Another aspect of the present application is a non-temporary computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the target detection method as described in the aforementioned aspect is implemented.

[0019] Another aspect of the present application provides a computer program product on which a computer program is stored. When the program is executed by a processor, the target detection method as described in the above aspect is implemented.

[0020] The target detection method, device, vehicle and storage medium proposed in the present application utilize the penetrability of sound waves to capture the sound of a specific frequency when a moving object (such as a pedestrian, a non-motor vehicle, a motor vehicle, etc.) moves through a microphone array installed on the vehicle, and then identify the target sound source (such as a moving object) in the vehicle's environment and locate the target sound source based on the sound wave signal captured by the microphone array. This can improve the accuracy and comprehensiveness of the identification of the target sound source (such as a moving object), avoid missing the identification of an obscured sound source or moving object, and cause a vehicle collision accident, thereby improving the safety of vehicle driving.

[0021] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A schematic diagram of a parking route for vehicles in an underground parking lot;

[0024] Figure 2 A schematic diagram of a flow chart of a first target detection method provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of a flow chart of a second target detection method provided in an embodiment of the present application;

[0026] Figure 4 A schematic diagram of a flow chart of a third target detection method provided in an embodiment of the present application;

[0027] Figure 5 A schematic diagram of a fourth target detection method provided in an embodiment of the present application;

[0028] Figure 6 A schematic diagram of the structure of a valet parking obstacle detection system based on an off-vehicle microphone array provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the target sound source recognition principle provided in an embodiment of the present application;

[0030] Figure 8 Schematic diagram of the relative position relationship between the vehicle and the target sound source S provided in the embodiment of the present application Figure 1 ;

[0031] Fig. 9 Schematic diagram of the relative position relationship between the vehicle and the target sound source S provided in the embodiment of the present application Figure 2 ;

[0032] Fig.10 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application;

[0033] Fig.11 is a block diagram of a vehicle according to an exemplary embodiment. DETAILED DESCRIPTION

[0034] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0035] Valet parking technology is a technical carrier to achieve the "last mile". Through the pre-made parking lot map or self-learning route map, the valet parking route is planned, and the vehicle is controlled to automatically drive along the predetermined route to the target parking position from the starting point of the route and park in the parking space at the target parking position. In the process of automatic driving of the vehicle, the vehicle-mounted sensing equipment (such as vehicle-mounted cameras, ultrasonic radars, laser radars, millimeter-wave radars, etc.) is used to detect obstacle information on the parking route in real time, and timely make decisions such as avoiding, detouring, avoiding, braking, etc.

[0036] As an example, the parking route of a vehicle in an underground parking lot can be as follows: Figure 1 As shown, the vehicle can automatically drive to the target parking position according to the planned parking route and park in the parking space at the target parking position.

[0037] However, the main scene of the valet parking function is in underground parking lots, and underground parking lots are characterized by many "ghost peeking" scenes caused by obstructions. The on-board sensing equipment cannot effectively detect moving objects (such as moving vehicles, pedestrians, etc.) that are blocked by obstructions (such as walls, etc.). When the moving objects are detected by the sensing equipment, the brakes are too late, resulting in a collision accident.

[0038] Among them, "ghost jumping" refers to when there is a vehicle or obstacle in front blocking the line of sight, that is, a blind spot, and a non-motor vehicle or pedestrian suddenly jumps out from the roadside.

[0039] In response to at least one of the above-mentioned problems, the present application proposes a target detection method, device, electronic device, vehicle and storage medium.

[0040] The target detection method, device, vehicle and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings.

[0041] Figure 2 A flowchart of the first target detection method provided in an embodiment of the present application.

[0042] The embodiment of the present application takes the target detection method being configured in a target detection device as an example. The target detection device can be applied to any electronic device or vehicle with computing capabilities so that the electronic device or vehicle can perform a target detection function.

[0043] The electronic device may be a personal computer, a mobile terminal, a television, etc. The mobile terminal may be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, or other hardware device with various operating systems.

[0044] like Figure 2 As shown, the target detection method may include the following steps S201 to S204:

[0045] Step S201, collecting sound wave signals in the environment where the vehicle is located through the microphone array of the vehicle.

[0046] Among them, a microphone array is installed on the vehicle, and the microphone array can be installed inside the vehicle or outside the vehicle, and the embodiment of the present application does not limit this.

[0047] The microphone array is composed of a plurality of microphones. It should be noted that the present application does not limit the installation positions of the plurality of microphones, and does not limit the installation distances between the plurality of microphones. For example, the plurality of microphones may be installed at equal intervals, or the plurality of microphones may be installed at unequal intervals. Exemplarily, the number of microphones in the microphone array is marked as N, where N is a positive integer greater than 1.

[0048] There is no restriction on the environment in which the vehicle is located. For example, the environment in which the vehicle is located may be an underground parking lot, a construction section, a narrow alley, a city street, etc.

[0049] In an embodiment of the present application, the vehicle's microphone array can be used to collect or capture sound wave signals in the vehicle's environment.

[0050] In a possible implementation of the embodiment of the present disclosure, in order to improve the accuracy of target detection, a microphone array installed outside the vehicle may be used to collect sound wave signals in the environment in which the vehicle is located.

[0051] Step S202: identifying a target sound source in the environment according to the sound wave signal.

[0052] The number of target sound sources may be one or more, and this is not limited in the embodiment of the present application.

[0053] Taking into account the different characteristic frequencies of motion sounds of different sound sources or moving objects (such as pure electric vehicles, hybrid vehicles, fuel vehicles, electric two-wheeled vehicles, pedestrians, etc.), for example, the characteristic frequency of pure electric vehicles is f1 Hz (Hertz), the characteristic frequency of hybrid vehicles is f2 Hz, the characteristic frequency of fuel vehicles is f3 Hz, the characteristic frequency of electric two-wheeled vehicles is f4 Hz, and so on, in this application, based on the characteristic frequencies of different sound sources, according to the sound wave signals, the sound sources (referred to as target sound sources in this application) in the environment where the vehicle is located can be identified.

[0054] Step S203, measuring the time difference between the sub-sound wave signal of the target sound source in the sound wave signal and reaching any two microphones in the microphone array.

[0055] In an embodiment of the present application, a sub-sound wave signal belonging to the target sound source can be determined from the sound wave signal according to the characteristic frequency of the target sound source. Exemplarily, the sound wave signal can be filtered and amplified based on the characteristic frequency of the target sound source to obtain the sub-sound wave signal of the target sound source.

[0056] In the embodiment of the present application, the time difference between the sub-sound wave signal reaching any two microphones in the microphone array can also be measured, that is, the time difference between the sound of the target sound source reaching any two microphones in the microphone array can be measured.

[0057] As an example, the sub-sound wave signal reaching the microphone M can be measured. i Time t i , and measure the sub-sound wave signal reaching the microphone M j Time t j , then the sub-sound wave signal reaches the microphone M i and M j The time difference is:

[0058] Δt ij =t i -t j ; (1)

[0059] Among them, i,j={1,2,3,…,N}.

[0060] Step S204: locate the current position of the target sound source according to the time difference and the installation positions of any two microphones.

[0061] In the embodiment of the present application, the current position of the target sound source can be calculated according to the time difference between any two microphones in the microphone array and the installation positions of the any two microphones.

[0062] As an example, first, a plane coordinate system (i.e., a vehicle coordinate system) is established with the center of the rear axle of the vehicle, wherein the origin of the vehicle coordinate system is the center of the rear axle, the positive direction of the horizontal axis (x-axis) is consistent with the driving direction of the vehicle, and the vertical axis (y-axis) is perpendicular to the x-axis. i The installation position (or installation coordinates) in the vehicle coordinate system is (x i ,y i ), microphone M j The installation location is (x j ,y j ), the current position of the target sound source S in the vehicle coordinate system is (x S ,y S ), target sound source S and microphone M i The straight-line distance between them is d i , target sound source S and microphone M j The straight-line distance between them is dj , then:

[0063]

[0064] Among them, c is the speed of sound, which is a constant, and i,j={1,2,3,…,N}.

[0065] Since (x i ,y i )、(x j ,y j ) and c are both fixed values, Δt ij is a measurable value. Therefore, the above formula (3) and formula (1) can be used to construct a set of equations to solve the current position (x S ,y S ).

[0066] The target detection method of the embodiment of the present application utilizes the penetrating property of sound waves to capture the sound of a specific frequency when a moving object (such as a pedestrian, a non-motor vehicle, a motor vehicle, etc.) moves through a microphone array installed on a vehicle, and then identifies the target sound source (such as a moving object) in the vehicle's environment and locates the position of the target sound source based on the sound wave signal captured by the microphone array. This can improve the accuracy and comprehensiveness of the identification of the target sound source (such as a moving object), avoid missing the identification of an obscured sound source or a moving object, and avoid the occurrence of a vehicle collision accident, thereby improving the safety of vehicle driving.

[0067] The present application embodiment provides another target detection method. Figure 3 A flowchart of the second target detection method provided in an embodiment of the present application.

[0068] It should be noted that the target detection method can be executed alone, or it can be executed in combination with any embodiment of the present application or a possible implementation method in the embodiment, or it can be executed in combination with any technical solution in the related technology, and the embodiments of the present application are not limited to this.

[0069] like Figure 3 As shown, the target detection method may include the following steps S301 to S306:

[0070] Step S301, collecting sound wave signals in the environment where the vehicle is located through the microphone array of the vehicle.

[0071] For the explanation of step S301, please refer to the relevant description in any embodiment of the present application, and will not be repeated here.

[0072] Step S302, obtaining a sound source frequency set; wherein the sound source frequency set includes characteristic frequencies of various types of known sound sources.

[0073] In an embodiment of the present application, sound samples of different types of known sound sources (such as pure electric vehicles, hybrid vehicles, fuel vehicles, electric two-wheeled vehicles, pedestrians, etc.) can be collected in advance, and spectral analysis can be performed on each sound sample to extract its main frequency components. Based on the spectral analysis results, the characteristic frequency (i.e., the main characteristic frequency) of each type of known sound source is determined, and the characteristic frequencies of these known sound sources are organized into a sound source frequency set.

[0074] As an example, if the number of known sound sources is n, the sound source frequency set can be: s ={f s1 ,f s2 ,f s3 ,…,f sn}, where f sk is the characteristic frequency of the kth known sound source, k = {1, 2, 3, …, n}.

[0075] It should be noted that the sound source frequency set can also be updated according to actual application requirements. For example, when a certain type of sound source is added to the traffic scene, the characteristic frequency of the sound source can be added to the sound source frequency set.

[0076] Step S303, obtaining the frequency spectrum of the sound wave signal.

[0077] In an embodiment of the present application, a related transformation algorithm (for example, Fourier transform (such as FFT (Fast Fourier Transform), STFT (Short-Time Fourier Transform), MFCCs (Mel-Frequency Cepstral Coefficients), gamma passband filter bank (Gammatone Filterbank), Bark algorithm (Bark band algorithm), etc.) can be used to transform the sound wave signal to obtain the spectrum of the sound wave signal.

[0078] Step S304: determining a target sound source in the environment from a plurality of known sound sources based on the sound source frequency set and the spectrum.

[0079] In the embodiment of the present application, the sound source frequency set and the spectrum of the sound wave signal can be integrated to determine the target sound source in the environment where the vehicle is located from a variety of known sound sources.

[0080] In any embodiment of the present application, the target sound source can be determined by following steps A to C:

[0081] Step A: Determine filtering frequency bands of multiple known sound sources based on the sound source frequency set.

[0082] As an example, the characteristic frequency of the k-th known sound source is marked as f sk , then the filtering frequency band of the kth known sound source can be: [f sk -Δ,f sk +Δ].

[0083] Here, Δ is a preset frequency interval, and illustratively, Δ may be 50 Hz.

[0084] Step B: filtering the frequency spectra based on the filtering frequency bands of the various known sound sources to obtain filtered frequency spectra corresponding to the various known sound sources.

[0085] Still using the above example, we can use a bandpass filter (filter frequency band is [f sk -Δ,f sk +Δ]), filter the spectrum of the sound wave signal to obtain the filtered spectrum corresponding to the kth known sound source.

[0086] Step C: performing energy detection on the filtered spectra corresponding to the multiple known sound sources respectively, so as to determine the target sound source from the multiple known sound sources according to the detection results.

[0087] In an embodiment of the present application, energy detection can be performed on the filtered spectra corresponding to multiple known sound sources to obtain detection results, wherein the detection results are used to indicate the energy of the multiple known sound sources. Therefore, in the present application, the target sound source in the vehicle's environment can be determined from the multiple known sound sources based on the detection results.

[0088] For example, a known sound source with relatively high energy may be used as a target sound source in the environment where the vehicle is located.

[0089] As an example, a target sound source may be determined from a plurality of known sound sources according to the detection result and a set energy threshold, wherein the energy of the target sound source is higher than the energy threshold.

[0090] Still taking the above example as an example, energy detection can be performed on the filtered spectrum corresponding to the kth known sound source to obtain the energy of the kth known sound source, and it is determined whether the energy of the kth known sound source is higher than the energy threshold. If so, the kth known sound source is used as the target sound source.

[0091] Step S305 , measuring the time difference between the sub-sound wave signal of the target sound source in the sound wave signal and reaching any two microphones in the microphone array.

[0092] Step S306: locate the current position of the target sound source according to the time difference and the installation positions of any two microphones.

[0093] The explanation of steps S305 to S306 can be found in the relevant description in any embodiment of the present application and will not be repeated here.

[0094] The target detection method of the embodiment of the present application can effectively identify a specific target sound source from a complex sound wave signal based on a sound source frequency set, and can be applied to various application scenarios, thereby improving the applicability of the method.

[0095] The present application embodiment provides another target detection method. Figure 4 A flowchart of the third target detection method provided in an embodiment of the present application.

[0096] It should be noted that the target detection method can be executed alone, or it can be executed in combination with any embodiment of the present application or a possible implementation method in the embodiment, or it can be executed in combination with any technical solution in the related technology, and the embodiments of the present application are not limited to this.

[0097] like Figure 4 As shown, the target detection method may include the following steps S401 to S407:

[0098] Step S401, collecting sound wave signals in the environment where the vehicle is located through the microphone array of the vehicle.

[0099] Step S402: identifying a target sound source in the environment according to the sound wave signal.

[0100] Step S403, measuring the time difference between the sub-sound wave signal of the target sound source in the sound wave signal and reaching any two microphones in the microphone array.

[0101] Step S404: locate the current position of the target sound source according to the time difference and the installation positions of any two microphones.

[0102] The explanation of steps S401 to S404 can be found in the relevant description in any embodiment of the present application and will not be repeated here.

[0103] Step S405, obtaining the initial receiving frequency and the current receiving frequency of the sub-sound wave signal.

[0104] In an embodiment of the present application, the initial receiving frequency of the sub-sound wave signal belonging to the target sound source in the sound wave signal can be obtained, and the frequency of the sub-sound wave signal received in real time during the movement of the target sound source can be obtained (referred to as the current receiving frequency in this application).

[0105] Step S406: Determine the current moving speed of the target sound source according to the initial receiving frequency and the current receiving frequency.

[0106] In the embodiment of the present application, the current movement speed (ie, real-time movement speed) of the target sound source may be calculated based on the Doppler effect, the initial receiving frequency, and the current receiving frequency.

[0107] As an example, the initial receiving frequency is marked as f0 and the current receiving frequency is marked as f Δ , the current speed of the target sound source is v, then:

[0108]

[0109] Among them, when v>0, it means that the target sound source is approaching the vehicle; when v<0, it means that the target sound source is moving away from the vehicle; when v=0, it means that the target sound source is stationary (that is, the target sound source and the vehicle are relatively stationary).

[0110] Step S407: Controlling the vehicle based on the current speed and position of the target sound source.

[0111] In the embodiment of the present application, the driving of the vehicle can be controlled based on the current movement speed and current position of the target sound source.

[0112] For example, when v>0, and / or the current position of the target sound source is close to the vehicle, the vehicle can be controlled to reduce its speed; when v≤0, and / or the current position of the target sound source is far from the vehicle, the vehicle can be controlled to continue driving in the current driving state.

[0113] The target detection method of the embodiment of the present application controls the vehicle's driving based on the relative position relationship and speed relationship between the target sound source and the vehicle. It can plan the vehicle to slow down or wait for the right of way before the vehicle enters a dangerous area, thereby avoiding collision accidents and improving the safety of vehicle driving.

[0114] The present application embodiment provides another target detection method. Figure 5 A flowchart of the fourth target detection method provided in an embodiment of the present application.

[0115] It should be noted that the target detection method can be executed alone, or it can be executed in combination with any embodiment of the present application or a possible implementation method in the embodiment, or it can be executed in combination with any technical solution in the related technology, and the embodiments of the present application are not limited to this.

[0116] like Figure 5 As shown, the target detection method may include the following steps S501 to S509:

[0117] Step S501, collecting sound wave signals in the environment where the vehicle is located through the microphone array of the vehicle.

[0118] Step S502: identifying a target sound source in the environment according to the sound wave signal.

[0119] Step S503: measuring the time difference between the sub-sound wave signal of the target sound source in the sound wave signal and the time difference between the sub-sound wave signal and any two microphones in the microphone array.

[0120] Step S504: locate the current position of the target sound source according to the time difference and the installation positions of any two microphones.

[0121] Step S505, obtaining the initial receiving frequency and the current receiving frequency of the sub-sound wave signal.

[0122] Step S506: Determine the current moving speed of the target sound source according to the initial receiving frequency and the current receiving frequency.

[0123] The explanation of steps S501 to S506 can be found in the relevant description in any embodiment of the present application and will not be repeated here.

[0124] Step S507: determining the movement trend of the target sound source according to the current movement speed.

[0125] In the embodiment of the present application, the movement trend of the target sound source can be determined according to the current movement speed of the target sound source.

[0126] As an example, when v>0, the movement trend of the target sound source is to approach the vehicle; when v<0, the movement trend of the target sound source is to move away from the vehicle; when v=0, the movement trend of the target sound source is to be relatively stationary with respect to the vehicle.

[0127] Step S508, obtaining the distance between the current position and the origin of the vehicle coordinate system.

[0128] In the embodiment of the present application, the distance between the current position of the target sound source and the origin of the vehicle coordinate system can be calculated. For example, the distance can be:

[0129] Step S509: Controlling the vehicle's movement according to at least one of the distance and the movement trend.

[0130] In the embodiment of the present application, the driving of the vehicle can be controlled according to at least one of the distance between the target sound source and the origin of the vehicle coordinate system and the movement trend of the target sound source.

[0131] In any embodiment of the present application, it can be determined whether the distance between the target sound source and the origin of the vehicle coordinate system is greater than a set first decision radius. When the distance is greater than the first decision radius, it indicates that the target sound source is far away from the vehicle. At this time, the vehicle can be controlled to continue driving in the current driving state.

[0132] Among them, the first decision radius is greater than the perception radius of the vehicle's perception device.

[0133] In any embodiment of the present application, when the distance between the target sound source and the origin of the vehicle coordinate system is less than or equal to the first decision radius, it can be further determined whether the distance is greater than the second decision radius; wherein the perception radius of the perception device < the second decision radius < the first decision radius.

[0134] If, that is, the second decision radius < the distance ≤ the first decision radius, it indicates that the target sound source is neither too close nor too far from the vehicle. At this time, it can be further determined whether the movement trend of the target sound source is approaching the vehicle. If the movement trend of the target sound source is approaching the vehicle (v>0), the vehicle is controlled to reduce the driving speed to the first vehicle speed threshold. If the movement trend of the target sound source is not approaching the vehicle (v≤0), that is, the movement trend of the target sound source is moving away from the vehicle or is relatively stationary with the vehicle, the vehicle is controlled to continue driving in the current driving state.

[0135] The first vehicle speed threshold refers to a pre-set safe vehicle speed.

[0136] In any embodiment of the present application, when the distance between the target sound source and the origin of the vehicle coordinate system is less than or equal to the second decision radius, it can be further determined whether the distance is greater than a third decision radius, wherein the third decision radius matches or is consistent with the perception radius of the vehicle's perception device.

[0137] If, that is, the third decision radius < the distance ≤ the second decision radius, it indicates that the target sound source is relatively close to the vehicle. At this time, it can be further determined whether the movement trend of the target sound source is approaching the vehicle. If the movement trend of the target sound source is approaching the vehicle (v>0), the vehicle is controlled to reduce the driving speed to the second vehicle speed threshold. If the movement trend of the target sound source is not approaching the vehicle (v≤0), that is, the movement trend of the target sound source is moving away from the vehicle or is relatively stationary with the vehicle, the vehicle is controlled to continue driving in the current driving state.

[0138] The second vehicle speed threshold is a pre-set vehicle speed threshold, and the second vehicle speed threshold is less than the first vehicle speed threshold. Exemplarily, the second vehicle speed threshold may be zero.

[0139] In any embodiment of the present application, when the distance between the target sound source and the origin of the vehicle coordinate system is less than or equal to the third decision radius, it indicates that the target sound source is already within the perception range of the vehicle's perception device. At this time, the perception result of the perception device can be obtained, and the vehicle can be controlled based on the perception result. At the same time, an alarm message can also be sent to remind relevant personnel (such as the driver, passengers, and remote control personnel who take over the vehicle in the background) that the target sound source is approaching the vehicle, and pay attention to avoid it in time.

[0140] The target detection method of the embodiment of the present application controls the vehicle's driving based on the relative position relationship and speed relationship between the target sound source and the vehicle. It can plan the vehicle to slow down or wait for the right of way before the vehicle enters a dangerous area, thereby avoiding collision accidents and improving the safety of vehicle driving.

[0141] Taking the installation of a microphone array on the outside of a vehicle as an example, the existing perception devices (such as vehicle-mounted cameras, ultrasonic radars, lidars, millimeter-wave radars, etc.) are unable to effectively solve the "ghost peeking" scenario during the automatic driving process of the valet parking function. This application proposes a valet parking obstacle detection system based on an external microphone array. By adding an external microphone to the vehicle and utilizing the penetrating nature of sound waves, the external microphone captures the sound of a moving object at a specific frequency when it is running, and calculates the relative position and distance relationship between the moving object and the vehicle through the Doppler principle, thereby achieving ultra-long-range detection of moving objects in the vehicle's perception blind spot, and planning the vehicle to slow down or wait for the right of way before entering the dangerous area, thereby avoiding vehicle collision accidents.

[0142] As an example, the structure of the valet parking obstacle detection system based on the external microphone array can be as follows: Figure 6 As shown, it mainly includes: a microphone array system 1 composed of multiple external microphones; a traditional sensing device 2 composed of a vehicle-mounted camera, an ultrasonic radar, a millimeter-wave radar, and a lidar; an automatic driving control system 3; a driving system 4; and a braking system 5.

[0143] Among them, the microphone array system 1 is mainly used to realize long-distance detection of obscured moving objects; the traditional sensing device 2 is mainly used to realize limited area detection during straight-line driving; the automatic driving control system 3 is mainly used to process the sensing output, plan routes, make decisions and control strategies, and send control instructions to the vehicle drive system 4 and braking system 5.

[0144] The following only describes the implementation of the ultra-long-distance detection of the microphone array system 1. The other technical implementations are well known to those skilled in the art and will not be described in detail here. The implementation of the ultra-long-distance detection by the microphone array system 1 mainly includes the following steps: 1. Identifying the target sound source; 2. Calculating the relative position and speed relationship between the target sound source and the vehicle; 3. Determining the driving strategy of the vehicle based on the kinematic relationship between the target sound source and the vehicle.

[0145] 1. Identify the target sound source.

[0146] The sound of different moving objects is distinguished by frequency, such as pure electric vehicles, hybrid vehicles, fuel vehicles, electric two-wheeled vehicles, etc., and the sound source frequency set f is established. s ={f s1 ,f s2 ,f s3 ,…,f sn}. When the microphone array receives the sound outside the car, Figure 7 The adaptive filter shown in the figure is used for the sound source frequency set f s The characteristic frequency in the sound wave signal is detected and amplified to identify the target sound source in the vehicle's environment from a variety of moving objects, and based on the characteristic frequency of the target sound source, the sound belonging to the target sound source is determined from the sound wave signal (referred to as a sub-sound wave signal in the present disclosure).

[0147] 2. Calculate the kinematic relationship between the target sound source and the vehicle (including relative position and velocity relationship).

[0148] First, establish the following with the center of the rear axle of the vehicle: Figure 8 The vehicle coordinate system shown in the figure assumes that the microphone array consists of 4 microphones, M i , i = {1, 2, 3, 4}, and their installation positions or installation coordinates in the vehicle coordinate system are (x i ,y i ), assuming that the current position of the target sound source S in the vehicle coordinate system is (x S ,y S ), the time it takes for the sound of the target sound source to reach each microphone is t i , the speed of sound is a constant c, and the time difference Δt between the sound of the target sound source and any two microphones ij is a measurable value, the microphone installation position (x i ,y i ) is also a fixed value, and the straight-line distance between the target sound source and any microphone is d i , the following formula can be used to calculate the relative position relationship between the target sound source and the vehicle:

[0149]

[0150]

[0151] where \(i,j = \{1,2,3,4\}\), \(\Delta t\) ij is a measurable value, \((x i ,y i ) is a determined value, and a system of equations can be constructed through the above equations to solve the current position \((x S ,y S ) of the target sound source.

[0152] Furthermore, according to the Doppler effect, the moving speed of the target sound source can be calculated. Assume that the initial received frequency of the sound of the target sound source is \(f_0\), the real-time received frequency during the movement is \(f Δ , and the moving speed of the target sound source is \(v\). Then, according to the Doppler effect, the moving speed of the target sound source can be calculated as follows:

[0153]

[0154] where when \(v>0\), it means that the target sound source is approaching the vehicle; when \(v<0\), it means that the target sound source is moving away from the vehicle; when \(v = 0\), it means that the target sound source is stationary (i.e., the target sound source is relatively stationary with respect to the vehicle).

[0155] 3. Determine the driving strategy of the host vehicle according to the kinematic relationship between the target sound source and the host vehicle.

[0156] Three decision radii can be set, namely the first decision radius \(R_1\), the second decision radius \(R_2\), and the third decision radius \(R_3\), where \(R_3<R_2<R_1\), and \(R_3\) coincides with the sensing radius of the traditional sensing device. If the calculated coordinate position of the target sound source \(S\) is outside \(R_1\) as shown in Fig. 9 , then control the host vehicle to maintain its current motion state; if the calculated coordinate position of the target sound source enters \(R_1\) but does not enter \(R_2\), and the motion trend is away from the host vehicle, then control the host vehicle to maintain its current motion state; if the calculated coordinate position of the target sound source enters \(R_1\) but does not enter \(R_2\), and the motion trend is towards the host vehicle, then reduce the vehicle speed to the safe vehicle speed \(v sf ; if the calculated coordinate position of the target sound source enters \(R_2\) but does not enter \(R_3\), and the motion trend is away from the host vehicle, then control the host vehicle to maintain its current motion state; if the calculated coordinate position of the target sound source enters \(R_2\) but does not enter \(R_3\), and the motion trend is towards the host vehicle, then reduce the vehicle speed to 0 and wait for other vehicles to pass; if the calculated coordinate position of the target sound source enters \(R_3\), then only send an alarm message, and the decision information only depends on the sensing result of the traditional sensing device.

[0157] In summary, the solution provided by this application has at least the following advantages:

[0158] 1) It can effectively solve the "ghost peeking" scenario in underground parking lots, improve the operational safety of valet parking functions, and enhance the property safety of users;

[0159] 2) It can enhance users’ confidence in intelligent driving functions and increase market penetration;

[0160] 3) Solve such long-tail scenarios and help the industry promote unmanned valet parking functions.

[0161] In order to implement the above embodiment, the embodiment of the present application also proposes a target detection device.

[0162] Fig.10 A schematic diagram of the structure of a target detection device provided in an embodiment of the present application.

[0163] like Fig.10 As shown, the target detection device 1000 may include: a collection module 1010 , a recognition module 1020 , a measurement module 1030 and a positioning module 1040 .

[0164] The acquisition module 1010 is used to collect sound wave signals in the environment where the vehicle is located through the microphone array of the vehicle;

[0165] An identification module 1020 is used to identify a target sound source in the environment according to the sound wave signal;

[0166] The measuring module 1030 is used to measure the time difference between the sub-sound wave signal of the target sound source in the sound wave signal and the time difference between the sub-sound wave signal and any two microphones in the microphone array;

[0167] The positioning module 1040 is used to locate the current position of the target sound source according to the time difference and the installation positions of any two microphones.

[0168] Furthermore, in an implementation of an embodiment of the present application, the identification module 1020 is specifically used to: obtain a sound source frequency set; wherein the sound source frequency set includes characteristic frequencies of multiple types of known sound sources; obtain a spectrum of the sound wave signal; and determine a target sound source in the environment from multiple known sound sources based on the sound source frequency set and the spectrum.

[0169] In one implementation of the embodiment of the present application, the identification module 1020 is specifically used to: determine the filtering frequency bands of multiple known sound sources based on the sound source frequency set; filter the frequency spectra based on the filtering frequency bands of the multiple known sound sources to obtain filtered frequency spectra corresponding to the multiple known sound sources; perform energy detection on the filtered frequency spectra corresponding to the multiple known sound sources, so as to determine the target sound source from the multiple known sound sources based on the detection results.

[0170] In one implementation of an embodiment of the present application, the identification module 1020 is specifically used to: perform energy detection on the filtered spectra corresponding to multiple known sound sources respectively to obtain detection results; wherein the detection results are used to indicate the energy of the multiple known sound sources; and determine the target sound source from the multiple known sound sources based on the detection results and the energy threshold; wherein the energy of the target sound source is higher than the energy threshold.

[0171] In one implementation of the embodiment of the present application, the target detection device 1000 may further include:

[0172] An acquisition module is used to acquire an initial receiving frequency and a current receiving frequency of a sub-sound wave signal of a target sound source in the sound wave signal;

[0173] A determination module, used to determine the current movement speed of the target sound source according to the initial receiving frequency and the current receiving frequency;

[0174] The control module is used to control the vehicle's driving based on the current moving speed and current position of the target sound source.

[0175] In one implementation of an embodiment of the present application, the control module is specifically used to: determine the movement trend of the target sound source based on the current movement speed; obtain the distance between the current position and the origin of the vehicle coordinate system; and control the vehicle's driving based on at least one of the distance and the movement trend.

[0176] In one implementation of the embodiment of the present application, the control module is specifically used to: when the distance is greater than a first decision radius, control the vehicle to continue driving in the current driving state; wherein the first decision radius is greater than the perception radius of the vehicle's perception device; when the distance is less than or equal to the first decision radius and greater than a second decision radius, determine whether the movement trend is approaching the vehicle; wherein the second decision radius is greater than the perception radius; if so, control the vehicle to reduce the driving speed to a first vehicle speed threshold; if not, control the vehicle to continue driving in the current driving state.

[0177] In one implementation of the embodiment of the present application, the control module is specifically used to: when the distance is less than or equal to the second decision radius and greater than the third decision radius, determine whether the movement trend is approaching the vehicle; wherein the third decision radius matches the perception radius of the vehicle's perception device; if so, control the vehicle to reduce the driving speed to a second vehicle speed threshold; wherein the second vehicle speed threshold is less than the first vehicle speed threshold; if not, control the vehicle to continue driving in the current driving state.

[0178] In one implementation of an embodiment of the present application, the control module is specifically used to: send an alarm message when the distance is less than or equal to a third decision radius; wherein the alarm message is used to indicate that the target sound source is close to the vehicle; obtain the perception result of the vehicle's perception device; and control the vehicle's driving based on the perception result.

[0179] It should be noted that the aforementioned explanation of the target detection method embodiment is also applicable to the target detection device of this embodiment, and will not be repeated here.

[0180] In the target detection device of the embodiment of the present application, the penetrability of sound waves is utilized to capture the sound of a specific frequency of a moving object (such as a pedestrian, a non-motor vehicle, a motor vehicle, etc.) when it moves through a microphone array installed on the vehicle. Then, based on the sound wave signal captured by the microphone array, the target sound source (such as a moving object) in the vehicle's environment and the position of the target sound source are identified. This can improve the accuracy and comprehensiveness of the identification of the target sound source (such as a moving object), avoid missing the identification of an obscured sound source or moving object, and avoid the occurrence of a vehicle collision accident, thereby improving the safety of vehicle driving.

[0181] In order to implement the above embodiments, the present application also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the target detection method as described in any of the above embodiments is implemented.

[0182] In order to implement the above embodiments, the present application also proposes a vehicle, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a target detection method as in any of the above embodiments.

[0183] In order to implement the above embodiments, the present application also proposes a non-temporary computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the target detection method as described in any of the above embodiments is implemented.

[0184] In order to implement the above embodiments, the present application also proposes a computer program product, on which a computer program is stored. When the computer program is executed by a processor, the target detection method as described in any of the above embodiments is implemented.

[0185] Fig.111 is a block diagram of a vehicle 1100 according to an exemplary embodiment. For example, the vehicle 1100 may be a hybrid vehicle, a non-hybrid vehicle, an electric vehicle, a fuel cell vehicle, or other types of vehicles. The vehicle 1100 may be an autonomous vehicle, a semi-autonomous vehicle, or a non-autonomous vehicle.

[0186] Reference Fig.11 , the vehicle 1100 may include various subsystems, such as an infotainment system 1110, a perception system 1120, a decision control system 1130, a drive system 1140, and a computing platform 1150. The vehicle 1100 may also include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and each component of the vehicle 1100 may be interconnected by wire or wireless means.

[0187] In some embodiments, the infotainment system 1110 may include a communication system, an entertainment system, and a navigation system, etc.

[0188] The perception system 1120 may include several sensors for sensing information about the environment around the vehicle 1100. For example, the perception system 1120 may include a global positioning system (the global positioning system may be a GPS system, or a Beidou system or other positioning systems), an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0189] The decision control system 1130 may include a computing system, a vehicle controller, a steering system, a throttle, and a braking system.

[0190] The drive system 1140 may include components that provide powered motion for the vehicle 1100. In one embodiment, the drive system 1140 may include an engine, an energy source, a transmission system, and wheels. The engine may be one or a combination of an internal combustion engine, an electric motor, and an air compression engine. The engine is capable of converting energy provided by the energy source into mechanical energy.

[0191] Some or all functions of the vehicle 1100 are controlled by a computing platform 1150. The computing platform 1150 may include at least one processor 1151 and a memory 1152, and the processor 1151 may execute instructions 1153 stored in the memory 1152.

[0192] The processor 1151 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processor (Graphic Process Unit, GPU), a field programmable gate array (Field Programmable Gate Array, FPGA), a system on chip (System on Chip, SOC), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC) or a combination thereof.

[0193] Memory 1152 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0194] In addition to instructions 1153 , memory 1152 may also store data, such as road maps, route information, and data such as the location, direction, and speed of the vehicle. The data stored in memory 1152 may be used by computing platform 1150 .

[0195] In the embodiment of the present application, the processor 1151 can execute instruction 1153 to complete all or part of the steps of the above-mentioned method embodiment.

[0196] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0197] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0198] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0199] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0200] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0201] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0202] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0203] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A target detection method, characterized in that: include: Collecting sound wave signals in the environment where the vehicle is located through a microphone array of the vehicle; identifying a target sound source in the environment according to the sound wave signal; Measuring the time difference between the sub-sound wave signal of the target sound source in the sound wave signal and the arrival of any two microphones in the microphone array; The current position of the target sound source is located according to the time difference and the installation positions of any two microphones.

2. The method according to claim 1, characterized in that: The step of identifying a target sound source in the environment according to the sound wave signal includes: Acquire a sound source frequency set; wherein the sound source frequency set includes characteristic frequencies of multiple types of known sound sources; Acquiring the frequency spectrum of the sound wave signal; The target sound source in the environment is determined from the plurality of known sound sources based on the sound source frequency set and the frequency spectrum.

3. The method according to claim 2, characterized in that The step of determining the target sound source in the environment from the plurality of known sound sources based on the sound source frequency set and the spectrum comprises: Determining filtering frequency bands of the plurality of known sound sources according to the sound source frequency set; Based on the filtering frequency bands of the multiple known sound sources, the frequency spectra are filtered to obtain filtered frequency spectra corresponding to the multiple known sound sources; Energy detection is performed on the filtered frequency spectra corresponding to the multiple known sound sources respectively, so as to determine the target sound source from the multiple known sound sources according to the detection results.

4. The method according to claim 3, characterized in that The performing energy detection on the filtered spectra corresponding to the multiple known sound sources respectively to determine the target sound source from the multiple known sound sources according to the detection results includes: Performing energy detection on the filtered spectra corresponding to the multiple known sound sources respectively to obtain detection results; wherein the detection results are used to indicate the energy of the multiple known sound sources; Determining the target sound source from the multiple known sound sources according to the detection result and the energy threshold; The energy of the target sound source is higher than the energy threshold.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Obtaining an initial receiving frequency and a current receiving frequency of the sub-sound wave signal; Determining a current moving speed of the target sound source according to the initial receiving frequency and the current receiving frequency; The vehicle is controlled to travel based on the current moving speed and the current position of the target sound source.

6. The method according to claim 5, characterized in that The step of controlling the vehicle based on the current movement speed and the current position of the target sound source includes: Determining a movement trend of the target sound source according to the current movement speed; Obtaining the distance between the current position and the origin of the vehicle coordinate system; The vehicle is controlled to travel according to at least one of the distance and the movement trend.

7. The method according to claim 6, characterized in that The step of controlling the vehicle according to at least one of the distance and the movement trend comprises: When the distance is greater than a first decision radius, controlling the vehicle to continue driving in the current driving state; wherein the first decision radius is greater than a perception radius of a perception device of the vehicle; When the distance is less than or equal to the first decision radius and greater than a second decision radius, determining whether the movement trend is approaching the vehicle; wherein the second decision radius is greater than the perception radius; If yes, controlling the vehicle to reduce the driving speed to a first vehicle speed threshold; If not, the vehicle is controlled to continue driving in the current driving state.

8. The method according to claim 6, characterized in that The step of controlling the vehicle according to at least one of the distance and the movement trend comprises: When the distance is less than or equal to the second decision radius and greater than the third decision radius, determining whether the movement trend is approaching the vehicle; wherein the third decision radius matches the perception radius of the perception device of the vehicle; If yes, control the vehicle to reduce the driving speed to a second vehicle speed threshold; wherein the second vehicle speed threshold is less than the first vehicle speed threshold; If not, the vehicle is controlled to continue driving in the current driving state.

9. The method according to claim 6, characterized in that The step of controlling the vehicle according to at least one of the distance and the movement trend comprises: When the distance is less than or equal to the third decision radius, sending an alarm message; wherein the alarm message is used to indicate that the target sound source is close to the vehicle; Acquiring a perception result of a perception device of the vehicle; Based on the perception result, driving control of the vehicle is performed.

10. A target detection device, characterized in that: include: A collection module, used to collect sound wave signals in the environment where the vehicle is located through a microphone array of the vehicle; an identification module, used to identify a target sound source in the environment according to the sound wave signal; A measuring module, used for measuring the time difference between the sub-sound wave signal of the target sound source in the sound wave signal and the arrival of any two microphones in the microphone array; A positioning module is used to locate the current position of the target sound source according to the time difference and the installation positions of any two microphones.

11. A vehicle, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 9 are implemented.

13. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

Images