Sound source sound intensity estimation method, device, equipment and readable storage medium

Abstract

The application discloses a sound source sound intensity estimation method, device and equipment and a computer readable storage medium. The method comprises the following steps: positioning the angle of a vehicle sound source relative to a microphone array based on sound signals collected by the microphone array and video data collected by a camera device; calculating the distance between the vehicle sound source and the microphone array according to the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array; and calculating the sound intensity data at the vehicle sound source according to the distance and the intensity data of the sound signals of the vehicle sound source collected by the microphone array. The application improves the accuracy of the estimated noise intensity of the vehicle sound source, thereby improving the monitoring and management accuracy of traffic noise.

Description

Technical Field

[0001] This invention relates to the field of vehicle monitoring technology, and in particular to a method, apparatus, device, and computer-readable storage medium for estimating the sound intensity of a sound source. Background Technology

[0002] With the rapid development of the urban economy, the number and speed of vehicles are increasing rapidly, leading to increasingly serious traffic noise, such as street racing and honking, which affects people's normal lives and work. As people's requirements for the living environment increase, the need for monitoring and managing traffic noise is becoming more and more urgent.

[0003] Identifying vehicles that generate traffic noise can be achieved by collecting sound signals using microphone arrays. The noise intensity generated by the vehicles can then be calculated from the sound signals, and management can be based on the noise intensity. However, the noise intensity estimated by microphone arrays is currently not accurate enough, resulting in low precision in the monitoring and management of traffic noise. Summary of the Invention

[0004] The main objective of this invention is to provide a method, apparatus, device, and computer-readable storage medium for estimating the sound intensity of a sound source. The aim is to propose a sound intensity estimation scheme for vehicle sound sources, improve the accuracy of the estimated noise intensity of vehicle sound sources, and thus facilitate the improvement of the accuracy of traffic noise monitoring and management.

[0005] To achieve the above objectives, the present invention provides a method for estimating the sound intensity of a sound source, the method comprising the following steps:

[0006] The angle of the vehicle's sound source relative to the microphone array is located based on the sound signals collected by the microphone array and the video data collected by the camera device.

[0007] The distance between the vehicle sound source and the microphone array is calculated based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array.

[0008] The sound intensity data at the vehicle sound source is calculated based on the distance and the sound signal intensity data of the vehicle sound source collected by the microphone array.

[0009] Optionally, the step of locating the angle of the vehicle sound source relative to the microphone array based on the sound signal collected by the microphone array and the video data collected by the camera device includes:

[0010] Vehicle identification is performed on the video data collected by the camera equipment to obtain the vehicle's position information within the field of view of the camera equipment;

[0011] The angular range of the vehicle relative to the microphone array is determined based on the location information;

[0012] Based on the sound signals collected by the microphone array, the sound sources within the specified angle range are located to obtain the angle of the sound sources relative to the microphone array within the specified angle range;

[0013] The sound source with the highest sound intensity is selected from the sound sources within the specified angle range as the vehicle sound source corresponding to the vehicle, so as to obtain the angle of the vehicle sound source relative to the microphone array, wherein the angle range is wider than the angle of the vehicle sound source relative to the microphone array.

[0014] Optionally, the sound source intensity estimation method further includes:

[0015] The sound signal collected by the microphone array is subjected to sound type detection. If the sound type of the sound signal collected by the microphone array is the target type, the step of performing vehicle identification on the video data collected by the camera device to obtain the vehicle's position information within the field of view of the camera device is executed.

[0016] Optionally, the step of detecting the sound type of the sound signal acquired by the microphone array includes:

[0017] Feature data is obtained by extracting features from the sound signals collected by the microphone array;

[0018] The feature data is input into a pre-trained classification model for classification to obtain the sound type of the sound signal collected by the microphone array.

[0019] Optionally, the angle of the vehicle sound source relative to the microphone array is represented by the pitch angle and azimuth angle of the vehicle sound source relative to the array coordinate system. The array coordinate system is a three-dimensional rectangular coordinate system established with a point in the plane where the microphone array is located as the origin, the plane where the microphone array is located as the xoy plane, and the normal of the xoy plane as the z-axis. The installation position information includes the installation angle and installation height of the microphone array. The installation angle is the angle between the plane where the microphone array is located and the normal of the installation ground. The installation height is the height of the origin relative to the installation ground.

[0020] The step of calculating the distance between the vehicle sound source and the microphone array based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array includes:

[0021] Based on the pitch and azimuth angles of the vehicle sound source relative to the microphone array, as well as the installation angle and height of the microphone array, the distance between the vehicle sound source and the microphone array is calculated using a distance calculation formula in a three-dimensional rectangular coordinate system.

[0022] Optionally, the sound intensity data is A-weighted sound pressure level, and the step of calculating the sound intensity data at the vehicle sound source based on the distance and the sound signal intensity data of the vehicle sound source collected by the microphone array includes:

[0023] Substituting the distance into the sound pressure level attenuation formula, the attenuation of the sound emitted by the vehicle sound source at the microphone array is calculated.

[0024] The A-weighted sound pressure level at the vehicle sound source is obtained by adding the attenuation amount to the A-weighted sound pressure level of the sound signal from the vehicle sound source collected by the microphone array.

[0025] Optionally, after the step of adding the attenuation amount to the A-weighted sound pressure level of the sound signal from the vehicle sound source acquired by the microphone array to obtain the A-weighted sound pressure level at the vehicle sound source, the method further includes:

[0026] The physical sound pressure level is obtained by mapping the A-weighted sound pressure level at the vehicle sound source using a preset mapping relationship.

[0027] To achieve the above objectives, the present invention also provides a sound source sound intensity estimation device, the device comprising:

[0028] The positioning module is used to locate the angle of the vehicle sound source relative to the microphone array based on the sound signals collected by the microphone array and the video data collected by the camera device.

[0029] The distance calculation module is used to calculate the distance between the vehicle sound source and the microphone array based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array.

[0030] The sound intensity calculation module is used to calculate the sound intensity data at the vehicle sound source based on the distance and the intensity data of the sound signal of the vehicle sound source collected by the microphone array.

[0031] To achieve the above objectives, the present invention also provides a sound source sound intensity estimation device, the sound source sound intensity estimation device comprising: a memory, a processor, and a sound source sound intensity estimation program stored in the memory and executable on the processor, wherein the sound source sound intensity estimation program, when executed by the processor, implements the steps of the sound source sound intensity estimation method as described above.

[0032] Furthermore, to achieve the above objectives, the present invention also proposes a computer-readable storage medium storing a sound source sound intensity estimation program, wherein the sound source sound intensity estimation program, when executed by a processor, implements the steps of the sound source sound intensity estimation method as described above.

[0033] In this embodiment of the invention, by combining the sound source localization function of the microphone array and the object localization function of the camera device, the angle of the vehicle sound source relative to the microphone array can be accurately located, thus laying a foundation for accurately estimating the sound intensity data of the sound emitted by the vehicle sound source at the vehicle sound source in subsequent steps. Then, by calculating the distance between the vehicle sound source and the microphone array based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array, and based on the distance and the sound signal intensity data of the vehicle sound source collected by the microphone array, the sound intensity data of the sound emitted by the vehicle sound source at the vehicle sound source is calculated. This sound intensity data can represent the true intensity of the sound emitted by the vehicle sound source and will not change due to changes in the monitoring position, thereby improving the accuracy of the estimated noise intensity of the vehicle sound source, and thus facilitating the improvement of the accuracy of traffic noise monitoring and management. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating an embodiment of the sound source intensity estimation method of the present invention;

[0035] Figure 2 This is a schematic diagram of the layout of a microphone array and camera device according to an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of an array coordinate system according to an embodiment of the present invention;

[0037] Figure 4 This is a flowchart illustrating a sound source sound pressure level estimation method according to an embodiment of the present invention.

[0038] Figure 5 This is a schematic diagram of the functional modules of a preferred embodiment of the sound source sound intensity estimation device of the present invention;

[0039] Figure 6 This is a schematic diagram of the hardware operating environment involved in the embodiments of the present invention.

[0040] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0041] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0042] Reference Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the sound source intensity estimation method of the present invention.

[0043] This invention provides an embodiment of a sound source sound intensity estimation method. It should be noted that although the flowchart shows a logical order, in some cases, the steps shown or described may be executed in a different order. In this embodiment, the executing entity of the sound source sound intensity estimation method can be a monitoring device, smartphone, personal computer, server, or other device; no limitation is imposed in this embodiment. For ease of description, a monitoring device is used as the executing entity in this embodiment. In this embodiment, the sound source sound intensity estimation method includes the following steps:

[0044] Step S10: Locate the angle of the vehicle sound source relative to the microphone array based on the sound signal collected by the microphone array and the video data collected by the camera device.

[0045] The monitoring equipment can include a microphone array and a camera. The equipment can be installed at monitoring points along the road to monitor noise from passing vehicles. In one feasible implementation, the center of the microphone array and the center of the camera can be at the same coordinate position. For example, as... Figure 2 As shown, the microphone array can include multiple microphones, which are distributed around the central origin in a circular array, and the camera device is set at the origin.

[0046] The microphone array and camera equipment can each collect data at a certain frequency. In this embodiment, the frequency at which the microphone array and camera equipment collect data is not limited. The monitoring equipment can estimate the sound intensity at a certain frequency. In this embodiment, the frequency at which the monitoring equipment estimates the sound intensity is also not limited. Furthermore, the frequency at which the microphone array and camera equipment collect data can be the same as or different from the frequency at which the monitoring equipment estimates the sound intensity. That is, the monitoring equipment can estimate the sound intensity at regular intervals based on one or more frames of audio signal recently collected by the microphone array and one or more frames of video data collected by the camera equipment.

[0047] A sound source is an object that emits sound; a vehicle sound source is a vehicle that emits sound. Monitoring equipment can locate the angle of the vehicle sound source relative to the microphone array based on sound signals collected by a microphone array and video data collected by a camera.

[0048] Microphone arrays can be used for sound source localization, that is, to locate the direction of a sound source. Video data collected by camera equipment can also be used to determine the location of vehicles through image recognition. In this embodiment, combining the localization functions of both allows the angle of the vehicle (a sound source) relative to the microphone array to be determined. There are many ways to combine the localization functions of the microphone array and camera equipment, and this embodiment does not impose any limitations. For example, in one feasible implementation, sound source localization can be performed using the sound signals collected by the microphone array to obtain the angles of each sound source relative to the microphone array. Then, the video data collected by the camera equipment is used to identify the angles of each vehicle relative to the microphone array. The monitoring equipment matches the angles of the vehicles relative to the microphone array with the angles of the sound sources relative to the microphone array. If the angle of a vehicle relative to the microphone array is the same as or within a certain error range as the angle of a sound source relative to the microphone array, then that vehicle is taken as the vehicle sound source, and the angle of that sound source relative to the microphone array is taken as the angle of the vehicle sound source relative to the microphone array.

[0049] In specific implementations, the sound source localization of the microphone array can be achieved using algorithms such as GCC-PHAT and MUSIC, and this embodiment does not impose any restrictions.

[0050] In one feasible implementation, the monitoring equipment can first calculate the intensity of the sound signal collected by the microphone array. If the calculated intensity is greater than a preset intensity, it indicates that noise with a significant impact on residents is occurring. The monitoring equipment then further locates the vehicle sound source based on the sound signal collected by the microphone array and the video data collected by the camera equipment, obtaining the angle of the vehicle sound source relative to the microphone array. If the calculated intensity is less than or equal to the preset intensity, it indicates that the noise has little impact on residents, and the monitoring equipment can skip locating the vehicle sound source, thus avoiding unnecessary location operations and wasting equipment resources.

[0051] It should be noted that there are many ways to represent the angle of the vehicle sound source relative to the microphone array, and this embodiment does not impose any limitations. For example, in one feasible implementation, a coordinate system can be established, and the azimuth and pitch angles in the coordinate system can be used to represent the angle of the vehicle sound source relative to the microphone array.

[0052] In this embodiment, by combining the sound source localization function of the microphone array and the object localization function of the camera device, the angle of the vehicle sound source relative to the microphone array can be accurately located, thus laying a foundation for accurately estimating the sound intensity data of the sound emitted by the vehicle sound source at the vehicle sound source in subsequent steps.

[0053] Step S20: Calculate the distance between the vehicle sound source and the microphone array based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array.

[0054] The monitoring equipment is mounted on the installation ground by a bracket. The installation location information can be information that can characterize the position of the microphone array relative to the installation ground. However, in this embodiment, it is not limited to which specific location information the installation location information of the microphone array includes. For example, in a feasible embodiment, the installation location information may include the installation height of the microphone array.

[0055] The monitoring equipment can calculate the distance between the vehicle sound source and the microphone array based on the installation location information of the microphone array and the angle of the vehicle sound source relative to the microphone array. It should be noted that the specific method for calculating the distance between the vehicle sound source and the microphone array may vary depending on the specific location information included in the installation location information and the specific representation of the angle of the vehicle sound source relative to the microphone array. This embodiment does not impose any restrictions or list them all.

[0056] Step S30: Calculate the sound intensity data at the vehicle sound source based on the distance and the sound signal intensity data of the vehicle sound source collected by the microphone array.

[0057] The sound signal from the vehicle sound source can be extracted from the sound signal collected by the microphone array, and the intensity data of the sound signal can be calculated according to the sound intensity calculation method. It should be noted that the specific representation of the sound signal intensity data is not limited in this embodiment; for example, it can be sound pressure level, sound power, etc.

[0058] It is understandable that the sound signal from the vehicle sound source collected by the microphone array is the sound signal that the sound emitted by the vehicle sound source travels to the location of the microphone array and is picked up by the microphone array. Therefore, its intensity data is the intensity data of the sound signal at the microphone array, rather than the intensity data of the sound emitted by the vehicle sound source at the vehicle sound source location.

[0059] In this embodiment, after calculating the distance between the vehicle sound source and the microphone array, the monitoring device can calculate the sound intensity data (hereinafter referred to as sound intensity data) at the vehicle sound source location based on this distance and the sound signal intensity data of the vehicle sound source collected by the microphone array. The attenuation of the sound signal intensity data can be calculated from the distance, and then the intensity data at the vehicle sound source location can be reconstructed by adding this attenuation to the sound signal intensity data collected by the microphone array. The calculation method may vary depending on the specific form of the intensity data, and this embodiment does not impose any limitations.

[0060] In one feasible implementation, step S30 includes:

[0061] Step S301: Substitute the distance into the sound pressure level attenuation formula to calculate the attenuation of the sound emitted by the vehicle sound source at the microphone array.

[0062] The sound intensity data can specifically be A-weighted sound pressure level. That is, in this embodiment, the monitoring device can calculate the A-weighted sound pressure level of the sound emitted by the vehicle sound source at the vehicle sound source location. The monitoring device can first substitute the distance into the sound pressure level attenuation formula to calculate the attenuation of the sound emitted by the vehicle sound source at the microphone array, that is, the amount of sound pressure level attenuation from the time the sound is emitted to the time it reaches the microphone array location.

[0063] The formula for sound pressure level attenuation is: Attenuation = 10log 10 R, where R represents the distance.

[0064] Step S302: Add the attenuation amount to the A-weighted sound pressure level of the sound signal from the vehicle sound source collected by the microphone array to obtain the A-weighted sound pressure level at the vehicle sound source.

[0065] The monitoring equipment can calculate the A-weighted sound pressure level (SPL) of the sound signal from the vehicle sound source captured by the microphone array. This APL represents the A-weighted sound level of the sound emitted by the vehicle sound source when it reaches the microphone array. By adding an attenuation factor to the APL of the sound signal from the vehicle sound source captured by the microphone array, the monitoring equipment can reconstruct the APL of the sound emitted by the vehicle sound source at the location of the vehicle sound source.

[0066] In one feasible implementation, after step S302, the method further includes:

[0067] Step S303: The A-weighted sound pressure level at the vehicle sound source is mapped to the physical sound pressure level using a preset mapping relationship.

[0068] Since the sound signal acquired by the microphone array is a digital signal, the calculated sound pressure level is the A-weighted sound pressure level in the digital signal sense (hereinafter referred to as S). A In specific application scenarios, the calculated digital signal sound pressure level can be further mapped to the physical sound pressure level (hereinafter referred to as S) as needed. A In general, a linear mapping is used, for example:

[0069] S A =a*S A '+b

[0070] Where a and b represent the parameters of the mapped line, which can be obtained by mapping the microphone signal collected in advance with the standard decibel meter value at the same location.

[0071] It should be noted that the road noise that can be perceived at the location of the microphone array does not represent the road noise that can be perceived by residents around the road. Therefore, in this embodiment, the sound intensity data of the sound emitted by the vehicle sound source at the vehicle sound source is calculated. This sound intensity data can represent the true intensity of the sound emitted by the vehicle sound source and will not change due to changes in the monitoring location, thereby improving the accuracy of the estimated noise intensity of the vehicle sound source, and thus helping to improve the accuracy of traffic noise monitoring and management.

[0072] In this embodiment, there is no limitation on how to perform noise management based on the sound intensity data obtained at the vehicle sound source.

[0073] In one feasible implementation, after acquiring the sound intensity data at the vehicle sound source, it can be detected whether the sound intensity data exceeds a set threshold. If it does, it indicates that the sound intensity emitted by the vehicle sound source is too high and may affect nearby residents. In this case, the location of the vehicle sound source in the video data can be identified to obtain the license plate number corresponding to the sound source, and then the license plate number can be recorded for targeted noise management based on the license plate number. It should be noted that since the angle and distance of the vehicle sound source relative to the microphone array are obtained, and the positional relationship between the microphone array and the camera equipment can be pre-calibrated, the angle and distance of the vehicle relative to the camera equipment can be calculated. This allows for the calculation of the vehicle's position in the video frame captured by the camera equipment. Based on this position, a certain range can be defined in the video frame for license plate recognition, accurately identifying the license plate number of the vehicle emitting the loudest noise and avoiding misidentification.

[0074] Based on the first embodiment described above, a second embodiment of the present invention for estimating the sound intensity of a sound source is proposed. In this embodiment, step S10 includes:

[0075] Step S101: Perform vehicle identification on the video data collected by the camera device to obtain the vehicle's position information within the field of view of the camera device.

[0076] In this embodiment, a specific implementation method is proposed to locate the angle of a vehicle sound source relative to a microphone by combining a microphone array and a camera device. Based on this implementation method, the efficiency and accuracy of vehicle sound source localization can be improved.

[0077] The monitoring equipment can perform vehicle identification on the video data collected by the camera equipment, specifically, vehicle identification on the captured video frame. The identification method is not limited in this embodiment and can be implemented using relevant image recognition algorithms. When a vehicle appears within the camera's field of view, the vehicle can be identified in the video frame, and its position information within the camera's field of view can be determined. When no vehicle appears within the camera's field of view, no vehicle can be identified in the video frame. In this case, the monitoring equipment can continue to identify the vehicle in the next frame of the video frame until it is identified.

[0078] It should be noted that, due to the size of the vehicle, the positioning granularity of the camera equipment is relatively coarse, so the calculated position information of the vehicle within the field of view of the camera equipment is not accurate enough, or it is just a position range.

[0079] Step S102: Determine the angular range of the vehicle relative to the microphone array based on the location information.

[0080] The relative positions of the microphone array and the camera equipment can be pre-calibrated. Based on this relative position and the vehicle's position within the camera equipment's field of view, the angular range of the vehicle relative to the microphone array can be calculated. In a specific implementation, when the vehicle's position within the camera equipment's field of view is a point, a position range can be obtained by expanding the range around that point. Based on this position range, an angular range can be calculated accordingly.

[0081] It should be noted that when multiple vehicles are in the field of view of the camera, there can be multiple calculated angle ranges, that is, each vehicle corresponds to a separate angle range.

[0082] Step S103: Based on the sound signals collected by the microphone array, locate the sound sources within the angle range to obtain the angle of the sound sources within the angle range relative to the microphone array.

[0083] After determining the angular range of the vehicle relative to the microphone array, the monitoring equipment can locate the sound source within that angular range based on the sound signals collected by the microphone array, thus obtaining the angle of the sound source relative to the microphone array within that angular range. Compared to locating the sound source at every angle, this embodiment only locates the sound source within the angular range of the vehicle relative to the microphone array, reducing the computational load and thereby improving the positioning efficiency.

[0084] In a specific implementation, after the monitoring equipment identifies a vehicle based on a single video frame, it can locate the sound source by simultaneously collecting sound signals from that video frame, thereby obtaining the angle of the sound source relative to the microphone array within the specified angle range.

[0085] Step S104: Select the sound source with the highest sound intensity from the sound sources within the angle range as the vehicle sound source corresponding to the vehicle, so as to obtain the angle of the vehicle sound source relative to the microphone array, wherein the angle range is wider than the angle of the vehicle sound source relative to the microphone array.

[0086] Locating sound sources within the angular range of the vehicle relative to the microphone array may result in one or more sound sources. When multiple sound sources exist within this range, their sound intensities can be compared, and the source with the highest intensity is identified as the vehicle's sound source. It's important to note that since the angular range of the vehicle relative to the microphone array is obtained through vehicle identification from video data, normally only the vehicle itself exists within this range, and it should be a relatively loud sound source. The presence of multiple sound sources indicates interference. In such cases, identifying the source with the highest intensity as the vehicle's sound source avoids interference and accurately locates the angle of the vehicle's sound source relative to the microphone array.

[0087] In one feasible embodiment, the sound source intensity estimation method further includes:

[0088] Step S40: Detect the sound type of the sound signal collected by the microphone array. If the sound type of the sound signal collected by the microphone array is the target type, then execute step S101.

[0089] The monitoring equipment can first perform sound type detection on the sound signals collected by the microphone array. The specific implementation method for sound type detection is not limited in this embodiment. Multiple sound types can be predefined, and the sound type requiring sound source localization (hereinafter referred to as the target type) can be defined. For example, the target type can be set to include street racing sounds, horn sounds, etc.

[0090] When the monitoring equipment detects that the sound type of the sound signal collected by the microphone array is any target type, it can further locate the angle of the vehicle sound source relative to the microphone array based on the sound signal collected by the microphone array and the video data collected by the camera equipment, and then calculate the distance based on the angle, and calculate the sound intensity data at the vehicle sound source based on the distance.

[0091] In this embodiment, the sound type of the sound signal collected by the microphone array is detected in advance. Only when the sound type is detected as the target type is the sound source intensity estimated. This avoids the situation where sound intensity is estimated when there is no noise on the road that affects residents, thereby reducing the equipment resource cost of the monitoring equipment for noise monitoring.

[0092] In one feasible implementation, step S40 includes:

[0093] Step S401: Feature data is obtained by extracting features from the sound signals collected by the microphone array.

[0094] This embodiment proposes a specific method for detecting the sound type of sound signals acquired by a microphone array. The monitoring device can extract feature data from the sound signals acquired by the microphone array, such as extracting Mel frequency cepstral coefficients, short-time zero-crossing rate, etc. Alternatively, a pre-trained feature extraction model can be used to extract feature data, such as training a Transformer, TDNN, or other network models to extract feature data from the sound signal.

[0095] Step S402: Input the feature data into a pre-trained classification model for classification to obtain the sound type of the sound signal collected by the microphone array.

[0096] The classification model can be pre-trained locally on the monitoring device or on other devices, and then deployed on the monitoring device. The classification model can be implemented using common types of classification models found in machine learning algorithms, and this implementation is not limited to any particular type. In one feasible implementation, the classification model and the feature extraction model can be trained together. The training method can be to pre-collect different types of sound signals, label each sound signal with its sound type, input each sound signal into the feature extraction model to be trained for feature extraction, obtain feature data, and then input the feature data into the classification model for classification, obtaining the predicted sound type corresponding to each sound signal. Based on the error between the predicted sound type and the sound type label for each sound signal, the model parameters in the feature extraction model and the classification model are updated. Through multiple rounds of iterative updates to the model parameters of the feature extraction model and the classification model, until a pre-set stopping condition is met, the trained feature extraction model and classification model are obtained and deployed on the monitoring device.

[0097] Based on the first and / or second embodiments described above, a third embodiment of the present invention for estimating the sound intensity of a sound source is proposed. In this embodiment, a specific implementation method for calculating the distance between a vehicle sound source and a microphone array is proposed.

[0098] The microphones in the microphone array can be arranged on a plane. A three-dimensional Cartesian coordinate system (hereinafter referred to as the array coordinate system) can be established with a point on the plane (e.g., the center of each microphone) as the origin, the plane as the xoy plane, and the normal to the xoy plane as the z-axis. The angle of the vehicle sound source relative to the microphone array can be represented by the pitch and azimuth angles of the vehicle sound source relative to this array coordinate system. The installation position information of the microphone array can include the installation angle and installation height. The installation angle can be the angle between the plane of the microphone array and the normal to the mounting ground, and the installation height can be the height of the origin of the array coordinate system relative to the mounting ground.

[0099] like Figure 3 As shown, H represents the mounting height of the microphone array, γ represents the mounting angle of the microphone array, and θ represents the pitch angle of the vehicle sound source relative to the array coordinate system. This represents the azimuth angle of the vehicle sound source relative to the array coordinate system. Figure 3 (Not shown in the image).

[0100] The monitoring equipment can calculate the distance between the vehicle sound source and the microphone array using the distance calculation formula in the three-dimensional rectangular coordinate system, based on the pitch and azimuth angles of the vehicle sound source relative to the microphone array, as well as the installation angle and height of the microphone array.

[0101] For example, in one feasible implementation, the distance between the vehicle sound source and the microphone array can be calculated using the following formula:

[0102]

[0103] In another feasible implementation, the coordinates of the vehicle sound source in the array coordinate system can be calculated first based on the pitch angle, azimuth angle, installation height and installation angle. Then, based on the coordinates of the vehicle sound source in the array coordinate system, the distance between the vehicle sound source and the origin can be calculated using the distance calculation formula between two points. This distance can be used as the distance between the vehicle sound source and the microphone array.

[0104] In one feasible implementation, such as Figure 4 As shown, a schematic diagram of a feasible process for estimating the sound pressure level at a vehicle sound source is presented.

[0105] Microphones [Mic_1, Mic_2, ..., Mic_N] in a microphone array collect sound signals [x_1(t), x_2(t), ..., x_N(t)]. Sound type detection is performed based on the sound signals collected by the microphone array. Vehicle detection is performed based on video data collected by a camera device. Based on the vehicle detection results, sound source localization is assisted to obtain the vehicle sound source (…). Figure 4The coordinates of the target sound source (which can be relative to a pre-established coordinate system, such as an array coordinate system or a ground coordinate system) can be used to calculate the distance between the vehicle sound source and the microphone array, based on the installation height and angle of the microphone array. Then, based on the distance and the sound pressure level of the sound signal from the vehicle sound source collected by the microphone array, the sound pressure level at the vehicle sound source location can be calculated. Figure 4 (The target sound pressure level is shown in the image.)

[0106] Furthermore, embodiments of the present invention also propose a sound source sound intensity estimation device, referring to... Figure 5 The device includes:

[0107] The positioning module 10 is used to locate the angle of the vehicle sound source relative to the microphone array based on the sound signals collected by the microphone array and the video data collected by the camera device.

[0108] The distance calculation module 20 is used to calculate the distance between the vehicle sound source and the microphone array based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array.

[0109] The sound intensity calculation module 30 is used to calculate the sound intensity data at the vehicle sound source based on the distance and the intensity data of the sound signal of the vehicle sound source collected by the microphone array.

[0110] In one feasible embodiment, the positioning module 10 is further configured to:

[0111] Vehicle identification is performed on the video data collected by the camera equipment to obtain the vehicle's position information within the field of view of the camera equipment;

[0112] The angular range of the vehicle relative to the microphone array is determined based on the location information;

[0113] Based on the sound signals collected by the microphone array, the sound sources within the specified angle range are located to obtain the angle of the sound sources relative to the microphone array within the specified angle range;

[0114] The sound source with the highest sound intensity is selected from the sound sources within the specified angle range as the vehicle sound source corresponding to the vehicle, so as to obtain the angle of the vehicle sound source relative to the microphone array, wherein the angle range is wider than the angle of the vehicle sound source relative to the microphone array.

[0115] In one feasible embodiment, the positioning module 10 is further configured to:

[0116] The sound signal collected by the microphone array is subjected to sound type detection. If the sound type of the sound signal collected by the microphone array is the target type, the operation of vehicle identification on the video data collected by the camera device is performed to obtain the vehicle's position information within the field of view of the camera device is executed.

[0117] In one feasible embodiment, the positioning module 10 is further configured to:

[0118] Feature data is obtained by extracting features from the sound signals collected by the microphone array;

[0119] The feature data is input into a pre-trained classification model for classification to obtain the sound type of the sound signal collected by the microphone array.

[0120] In one feasible embodiment, the angle of the vehicle sound source relative to the microphone array is represented by the pitch angle and azimuth angle of the vehicle sound source relative to the array coordinate system. The array coordinate system is a three-dimensional rectangular coordinate system established with a point in the plane where the microphone array is located as the origin, the plane where the microphone array is located as the xoy plane, and the normal of the xoy plane as the z-axis. The installation position information includes the installation angle and installation height of the microphone array. The installation angle is the angle between the plane where the microphone array is located and the normal of the installation ground. The installation height is the height of the origin relative to the installation ground.

[0121] The distance calculation module 20 is also used for:

[0122] Based on the pitch and azimuth angles of the vehicle sound source relative to the microphone array, as well as the installation angle and height of the microphone array, the distance between the vehicle sound source and the microphone array is calculated using a distance calculation formula in a three-dimensional rectangular coordinate system.

[0123] In one feasible embodiment, the sound intensity calculation module 30 is further configured to:

[0124] Substituting the distance into the sound pressure level attenuation formula, the attenuation of the sound emitted by the vehicle sound source at the microphone array is calculated.

[0125] The A-weighted sound pressure level at the vehicle sound source is obtained by adding the attenuation amount to the A-weighted sound pressure level of the sound signal from the vehicle sound source collected by the microphone array.

[0126] In one feasible embodiment, after adding the attenuation amount to the A-weighted sound pressure level of the sound signal from the vehicle sound source acquired by the microphone array to obtain the A-weighted sound pressure level at the vehicle sound source, the sound intensity calculation module 30 is further configured to:

[0127] The physical sound pressure level is obtained by mapping the A-weighted sound pressure level at the vehicle sound source using a preset mapping relationship.

[0128] Furthermore, embodiments of the present invention also propose a sound source sound intensity estimation device, such as... Figure 6 As shown, Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of the present invention. It should be noted that the sound source intensity estimation device in the embodiments of the present invention can be a monitoring device, a smartphone, a personal computer, a server, etc., and no specific limitation is made here.

[0129] like Figure 6 As shown, the sound source intensity estimation device may include: a processor 1001, such as a CPU; a network interface 1004; a user interface 1003; a memory 1005; and a communication bus 1002. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0130] Those skilled in the art will understand that Figure 6 The device structure shown does not constitute a limitation on the sound source intensity estimation device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0131] like Figure 6 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and a sound source intensity estimation program. The operating system is a program that manages and controls the device's hardware and software resources, supporting the operation of the sound source intensity estimation program and other software or programs. Figure 6 In the device shown, the user interface 1003 is mainly used for data communication with the client; the network interface 1004 is mainly used for establishing a communication connection with the server; and the processor 1001 can be used to call the sound source intensity estimation program stored in the memory 1005 and perform the following operations:

[0132] The angle of the vehicle's sound source relative to the microphone array is located based on the sound signals collected by the microphone array and the video data collected by the camera device.

[0133] The distance between the vehicle sound source and the microphone array is calculated based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array.

[0134] The sound intensity data at the vehicle sound source is calculated based on the distance and the sound signal intensity data of the vehicle sound source collected by the microphone array.

[0135] In one feasible implementation, the operation of locating the angle of the vehicle sound source relative to the microphone array based on the sound signal acquired by the microphone array and the video data acquired by the camera device includes:

[0136] Vehicle identification is performed on the video data collected by the camera equipment to obtain the vehicle's position information within the field of view of the camera equipment;

[0137] The angular range of the vehicle relative to the microphone array is determined based on the location information;

[0138] Based on the sound signals collected by the microphone array, the sound sources within the specified angle range are located to obtain the angle of the sound sources relative to the microphone array within the specified angle range;

[0139] The sound source with the highest sound intensity is selected from the sound sources within the specified angle range as the vehicle sound source corresponding to the vehicle, so as to obtain the angle of the vehicle sound source relative to the microphone array, wherein the angle range is wider than the angle of the vehicle sound source relative to the microphone array.

[0140] In one feasible embodiment, the processor 1001 can also be used to call a sound source intensity estimation program stored in the memory 1005 and perform the following operations:

[0141] The sound signal collected by the microphone array is subjected to sound type detection. If the sound type of the sound signal collected by the microphone array is the target type, the operation of vehicle identification on the video data collected by the camera device is performed to obtain the vehicle's position information within the field of view of the camera device is executed.

[0142] In one feasible implementation, the operation of detecting the sound type of the sound signal acquired by the microphone array includes:

[0143] Feature data is obtained by extracting features from the sound signals collected by the microphone array;

[0144] The feature data is input into a pre-trained classification model for classification to obtain the sound type of the sound signal collected by the microphone array.

[0145] In one feasible embodiment, the angle of the vehicle sound source relative to the microphone array is represented by the pitch angle and azimuth angle of the vehicle sound source relative to the array coordinate system. The array coordinate system is a three-dimensional rectangular coordinate system established with a point in the plane where the microphone array is located as the origin, the plane where the microphone array is located as the xoy plane, and the normal of the xoy plane as the z-axis. The installation position information includes the installation angle and installation height of the microphone array. The installation angle is the angle between the plane where the microphone array is located and the normal of the installation ground. The installation height is the height of the origin relative to the installation ground.

[0146] The operation of calculating the distance between the vehicle sound source and the microphone array based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array includes:

[0147] Based on the pitch and azimuth angles of the vehicle sound source relative to the microphone array, as well as the installation angle and height of the microphone array, the distance between the vehicle sound source and the microphone array is calculated using a distance calculation formula in a three-dimensional rectangular coordinate system.

[0148] In one feasible implementation, the sound intensity data is an A-weighted sound pressure level, and the operation of calculating the sound intensity data at the vehicle sound source based on the distance and the sound signal intensity data of the vehicle sound source collected by the microphone array includes:

[0149] Substituting the distance into the sound pressure level attenuation formula, the attenuation of the sound emitted by the vehicle sound source at the microphone array is calculated.

[0150] The A-weighted sound pressure level at the vehicle sound source is obtained by adding the attenuation amount to the A-weighted sound pressure level of the sound signal from the vehicle sound source collected by the microphone array.

[0151] In one feasible implementation, after adding the attenuation amount to the A-weighted sound pressure level of the sound signal from the vehicle sound source acquired by the microphone array to obtain the A-weighted sound pressure level at the vehicle sound source, the processor 1001 can also be used to call the sound source sound intensity estimation program stored in the memory 1005 to perform the following operations:

[0152] The physical sound pressure level is obtained by mapping the A-weighted sound pressure level at the vehicle sound source using a preset mapping relationship.

[0153] Furthermore, embodiments of the present invention also propose a computer-readable storage medium storing a sound source sound intensity estimation program, wherein when the sound source sound intensity estimation program is executed by a processor, it implements the steps of the sound source sound intensity estimation method described below.

[0154] The various embodiments of the sound source sound intensity estimation device and computer-readable storage medium of the present invention can all refer to the various embodiments of the sound source sound intensity estimation method of the present invention, and will not be described again here.

[0155] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0156] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0157] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0158] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of protection of the present invention.

Claims

1. A method for estimating the sound intensity of a sound source, characterized in that, The sound source intensity estimation method includes the following steps: The angle of the vehicle's sound source relative to the microphone array is located based on the sound signals collected by the microphone array and the video data collected by the camera device. The distance between the vehicle sound source and the microphone array is calculated based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array. Based on the distance and the intensity data of the sound signal from the vehicle sound source collected by the microphone array, the sound intensity data at the vehicle sound source is calculated. The step of locating the angle of the vehicle sound source relative to the microphone array based on the sound signal collected by the microphone array and the video data collected by the camera device includes: Vehicle identification is performed on the video data collected by the camera equipment to obtain the vehicle's position information within the field of view of the camera equipment; The angular range of the vehicle relative to the microphone array is determined based on the location information; Based on the sound signals collected by the microphone array, the sound sources within the specified angle range are located to obtain the angle of the sound sources relative to the microphone array within the specified angle range; The sound source with the highest sound intensity is selected from the sound sources within the specified angle range as the vehicle sound source corresponding to the vehicle, so as to obtain the angle of the vehicle sound source relative to the microphone array, wherein the angle range is wider than the angle of the vehicle sound source relative to the microphone array.

2. The sound source intensity estimation method as described in claim 1, characterized in that, The sound source intensity estimation method further includes: The sound signal collected by the microphone array is subjected to sound type detection. If the sound type of the sound signal collected by the microphone array is the target type, the step of performing vehicle identification on the video data collected by the camera device to obtain the vehicle's position information within the field of view of the camera device is executed.

3. The sound source intensity estimation method as described in claim 2, characterized in that, The step of detecting the sound type of the sound signal acquired by the microphone array includes: Feature data is obtained by extracting features from the sound signals collected by the microphone array; The feature data is input into a pre-trained classification model for classification to obtain the sound type of the sound signal collected by the microphone array.

4. The sound source intensity estimation method as described in claim 1, characterized in that, The angle of the vehicle sound source relative to the microphone array is represented by the pitch angle and azimuth angle of the vehicle sound source relative to the array coordinate system. The array coordinate system is a three-dimensional rectangular coordinate system established with a point in the plane where the microphone array is located as the origin, the plane where the microphone array is located as the xoy plane, and the normal of the xoy plane as the z-axis. The installation position information includes the installation angle and installation height of the microphone array. The installation angle is the angle between the plane where the microphone array is located and the normal of the installation ground. The installation height is the height of the origin relative to the installation ground. The step of calculating the distance between the vehicle sound source and the microphone array based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array includes: Based on the pitch and azimuth angles of the vehicle sound source relative to the microphone array, as well as the installation angle and height of the microphone array, the distance between the vehicle sound source and the microphone array is calculated using a distance calculation formula in a three-dimensional rectangular coordinate system.

5. The sound source intensity estimation method according to any one of claims 1 to 4, characterized in that, The sound intensity data is A-weighted sound pressure level. The step of calculating the sound intensity data at the vehicle sound source based on the distance and the sound signal intensity data of the vehicle sound source collected by the microphone array includes: Substituting the distance into the sound pressure level attenuation formula, the attenuation of the sound emitted by the vehicle sound source at the microphone array is calculated. The A-weighted sound pressure level at the vehicle sound source is obtained by adding the attenuation amount to the A-weighted sound pressure level of the sound signal from the vehicle sound source collected by the microphone array.

6. The sound source intensity estimation method as described in claim 5, characterized in that, After the step of adding the attenuation amount to the A-weighted sound pressure level of the sound signal from the vehicle sound source acquired by the microphone array to obtain the A-weighted sound pressure level at the vehicle sound source, the method further includes: The physical sound pressure level is obtained by mapping the A-weighted sound pressure level at the vehicle sound source using a preset mapping relationship.

7. A sound source sound intensity estimation device, characterized in that, The sound source intensity estimation device includes: The positioning module is used to locate the angle of the vehicle sound source relative to the microphone array based on the sound signals collected by the microphone array and the video data collected by the camera device. The distance calculation module is used to calculate the distance between the vehicle sound source and the microphone array based on the angle of the vehicle sound source relative to the microphone array and the installation position information of the microphone array. The sound intensity calculation module is used to calculate the sound intensity data at the vehicle sound source based on the distance and the intensity data of the sound signal from the vehicle sound source collected by the microphone array; The sound source intensity estimation device is also used to achieve: Vehicle identification is performed on the video data collected by the camera equipment to obtain the vehicle's position information within the field of view of the camera equipment; The angular range of the vehicle relative to the microphone array is determined based on the location information; Based on the sound signals collected by the microphone array, the sound sources within the specified angle range are located to obtain the angle of the sound sources relative to the microphone array within the specified angle range; The sound source with the highest sound intensity is selected from the sound sources within the specified angle range as the vehicle sound source corresponding to the vehicle, so as to obtain the angle of the vehicle sound source relative to the microphone array, wherein the angle range is wider than the angle of the vehicle sound source relative to the microphone array.

8. A sound source intensity estimation device, characterized in that, The sound source intensity estimation device includes: a memory, a processor, and a sound source intensity estimation program stored in the memory and executable on the processor. When the sound source intensity estimation program is executed by the processor, it implements the steps of the sound source intensity estimation method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a sound source sound intensity estimation program, which, when executed by a processor, implements the steps of the sound source sound intensity estimation method as described in any one of claims 1 to 6.

Images