Acoustic imaging frequency determination method, apparatus, device, and readable storage medium

Abstract

The application discloses an acoustic imaging frequency determination method, device and equipment and a readable storage medium. Generally, the position of a target sound source may appear is specific. Therefore, in the application, a preset region of interest is first determined based on the position of the target sound source, and then a preliminary imaging frequency range of the target sound source is determined based on audio data corresponding to the preset region of interest, so that the calculation workload and power consumption are reduced, and the frequency determination efficiency is improved. After the preliminary imaging frequency range of the target sound source is determined, an acoustic imaging image including a sound field heat map corresponding to each preliminary imaging frequency range, a video image and the region of interest is generated in combination with the video image for each preliminary imaging frequency range, and an effective imaging frequency range of the target sound source is determined based on the position relationship between the sound field heat map corresponding to each preliminary imaging frequency range and the region of interest, so that the determined frequency range of the target sound source has higher accuracy.

Description

Technical Field

[0001] This application relates to the field of acoustic imaging technology, and more specifically, to an acoustic imaging frequency determination method, apparatus, device, and readable storage medium. Background Technology

[0002] Acoustic imaging is a measurement technique based on acoustic sensor arrays (such as microphone arrays). It measures the phase difference of sound waves arriving at each acoustic sensor within a defined space, determines the location of the sound source based on the phased array principle, measures the amplitude of the sound source, and displays the spatial distribution of the sound source as an acoustic image, where color and brightness represent intensity. By overlaying the acoustic image of a given space with a video image, the distribution of the sound source in that space can be visually displayed, thus helping people quickly locate the sound source. Acoustic imaging technology has been widely applied in several fields, such as rapidly detecting pressurized gas leaks and vacuum leaks in industrial settings, and quickly identifying potential partial discharge faults in power systems.

[0003] In reality, sound is often multi-source, meaning that the sound heard at the same time may come from the superposition of different sound sources. The manifestation of these different sound sources in acoustic imaging is reflected in the sound field heat map containing multiple sound sources in the acoustic image. However, in practical application scenarios, it is often necessary to locate only the target sound source. Since different sound sources have different emission frequencies, the frequency range of the target sound source can be determined, and imaging of only the target sound source can be achieved by adjusting the frequency.

[0004] In existing technologies, the frequency range of the target sound source is usually determined manually by the user, which requires the user to have certain experience and is prone to errors in determining the frequency range of the target sound source. Some existing methods for determining the frequency range of the target sound source are also relatively inefficient.

[0005] Therefore, how to provide an efficient and reliable method for determining the frequency range of a target sound source has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] In view of the above problems, this application proposes a method, apparatus, device, and readable storage medium for determining acoustic imaging frequency. The specific solution is as follows:

[0007] An acoustic imaging frequency determination method, the method comprising:

[0008] Based on the audio data corresponding to the preset region of interest, the preliminary imaging frequency range of the target sound source is determined; the region of interest is a region in the video image to be acoustically imaged, and the area of ​​the region of interest is smaller than the area of ​​the video image.

[0009] For each of the aforementioned preliminary imaging frequency ranges, an acoustic imaging map is generated by combining the video image. The acoustic imaging map includes a sound field heat map corresponding to each of the aforementioned preliminary imaging frequency ranges, the video image, and the region of interest.

[0010] Based on the positional relationship between the acoustic field heatmaps corresponding to each of the preliminary imaging frequency ranges and the region of interest, the effective imaging frequency range of the target sound source is determined.

[0011] Optionally, determining the preliminary imaging frequency range of the target sound source based on the audio data corresponding to the preset region of interest includes:

[0012] A fast Fourier transform is performed on the audio data corresponding to the preset region of interest to obtain a spectrogram, which is used to indicate the sound source energy value at each preset frequency point;

[0013] The first screening threshold is determined based on the sound source energy value at each preset frequency point;

[0014] Based on the spectrogram and the first screening threshold, the preliminary imaging frequency range of the target sound source is determined.

[0015] Optionally, determining the effective imaging frequency range of the target sound source based on the positional relationship between the acoustic field heatmaps corresponding to each of the preliminary imaging frequency ranges and the region of interest includes:

[0016] The preliminary imaging frequency range corresponding to the sound field heatmap within the region of interest is determined as the effective imaging frequency range of the target sound source;

[0017] The preliminary imaging frequency range corresponding to the acoustic field thermogram outside the region of interest is determined to be the invalid imaging frequency range.

[0018] Optionally, after determining the effective imaging frequency range of the target sound source based on the positional relationship between the sound field heatmaps corresponding to each of the preliminary imaging frequency ranges and the region of interest, the method further includes:

[0019] Based on the acoustic field thermograms corresponding to each of the preliminary imaging frequency ranges, the recommended imaging frequency range is determined from the effective imaging frequency range.

[0020] Calculate the distance between the acoustic field heatmap corresponding to each of the recommended imaging frequency ranges and the region of interest;

[0021] According to the rule that the closer the distance, the higher the priority, the priority of each of the recommended imaging frequency ranges is sorted based on the distance between the sound field heatmap corresponding to each of the recommended imaging frequency ranges and the region of interest, so as to obtain the sorted imaging frequency ranges to be recommended.

[0022] The sorted range of imaging frequencies to be recommended is then presented to the user.

[0023] Optionally, determining the recommended imaging frequency range from the effective imaging frequency range based on the acoustic field thermograms corresponding to each of the preliminary imaging frequency ranges includes:

[0024] Calculate the distance between every two adjacent acoustic field heatmaps;

[0025] If the distance between two adjacent sound field heatmaps is less than a preset distance threshold, then the sound source energy value of the effective imaging frequency range corresponding to the two adjacent sound field heatmaps is obtained.

[0026] Among the effective imaging frequency ranges corresponding to two adjacent sound field heatmaps, the effective imaging frequency range with the larger sound source energy value is determined as the recommended imaging frequency range.

[0027] If the distance between two adjacent acoustic field heatmaps is greater than a preset distance threshold, then the effective imaging frequency range corresponding to the two adjacent acoustic field heatmaps is determined as the recommended imaging frequency range.

[0028] Optionally, after recommending the sorted imaging frequency ranges to the user, if an instruction to further filter frequency bands is received, the method further includes:

[0029] Determine a second screening threshold, wherein the second screening threshold is less than the first screening threshold;

[0030] Based on the spectrogram and the second screening threshold, the new imaging frequency range of the target sound source is determined;

[0031] The newly added imaging frequency range is used as a new preliminary imaging frequency range for processing until the imaging frequency range of the target sound source is determined.

[0032] Optionally, determining the new imaging frequency range of the target sound source based on the spectrogram and the second screening threshold includes:

[0033] Based on the spectrum, at least one undetermined frequency band whose sound source energy value exceeds the second screening threshold is determined;

[0034] Among at least one specific frequency band that exceeds the second screening threshold, after removing at least one specific frequency band that exceeds the first screening threshold, the remaining specific frequency bands are the new imaging frequency range of the target sound source.

[0035] An acoustic imaging frequency determination device, the device comprising:

[0036] The preliminary imaging frequency range determination unit is used to determine the preliminary imaging frequency range of the target sound source based on the audio data corresponding to the preset region of interest; the region of interest is a region in the video image to be acoustically imaged, and the area of ​​the region of interest is smaller than the area of ​​the video image.

[0037] An acoustic imaging map determination unit is used to generate an acoustic imaging map for each of the preliminary imaging frequency ranges, in combination with the video image. The acoustic imaging map includes a sound field heat map corresponding to each of the preliminary imaging frequency ranges, the video image, and the region of interest.

[0038] The effective imaging frequency range determination unit is used to determine the effective imaging frequency range of the target sound source based on the positional relationship between the sound field heat map corresponding to each of the preliminary imaging frequency ranges and the region of interest.

[0039] Optionally, the preliminary imaging frequency range determination unit includes:

[0040] The spectrum determination subunit is used to perform a fast Fourier transform on the audio data corresponding to the preset region of interest to obtain a spectrum, which is used to indicate the sound source energy value at each preset frequency point;

[0041] The first screening threshold determination subunit is used to determine the first screening threshold based on the sound source energy value at each preset frequency point;

[0042] The preliminary imaging frequency range determination subunit is used to determine the preliminary imaging frequency range of the target sound source based on the spectrum and the first screening threshold.

[0043] Optionally, the effective imaging frequency range determination unit is specifically used for:

[0044] The preliminary imaging frequency range corresponding to the sound field heatmap within the region of interest is determined as the effective imaging frequency range of the target sound source;

[0045] The preliminary imaging frequency range corresponding to the acoustic field thermogram outside the region of interest is determined to be the invalid imaging frequency range.

[0046] Optionally, the device further includes:

[0047] The unit for determining the recommended imaging frequency range is used to determine the recommended imaging frequency range from the effective imaging frequency range based on the sound field heat map corresponding to each of the preliminary imaging frequency ranges and the positional relationship between the sound field heat map and the region of interest, after determining the effective imaging frequency range of the target sound source.

[0048] The distance calculation unit is used to calculate the distance between the acoustic field heatmap corresponding to each of the recommended imaging frequency ranges and the region of interest.

[0049] The sorting unit is used to sort the priority of each of the recommended imaging frequency ranges according to the rule that the closer the distance, the higher the priority, based on the distance between the acoustic field heat map corresponding to each of the recommended imaging frequency ranges and the region of interest, so as to obtain the sorted imaging frequency ranges to be recommended.

[0050] The recommendation unit is used to recommend the sorted imaging frequency range to the user.

[0051] Optionally, the unit for determining the recommended imaging frequency range is specifically used for:

[0052] Calculate the distance between every two adjacent acoustic field heatmaps;

[0053] If the distance between two adjacent sound field heatmaps is less than a preset distance threshold, then the sound source energy value of the effective imaging frequency range corresponding to the two adjacent sound field heatmaps is obtained.

[0054] Among the effective imaging frequency ranges corresponding to two adjacent sound field heatmaps, the effective imaging frequency range with the larger sound source energy value is determined as the recommended imaging frequency range.

[0055] If the distance between two adjacent acoustic field heatmaps is greater than a preset distance threshold, then the effective imaging frequency range corresponding to the two adjacent acoustic field heatmaps is determined as the recommended imaging frequency range.

[0056] Optionally, after recommending the sorted range of imaging frequencies to the user, if an instruction to further filter frequency bands is received, the device further includes:

[0057] The second screening threshold determination unit is used to determine a second screening threshold, wherein the second screening threshold is less than the first screening threshold;

[0058] A new imaging frequency range determination unit is added, which is used to determine the new imaging frequency range of the target sound source based on the spectrum and the second screening threshold.

[0059] The processing unit is used to process the newly added imaging frequency range as a new preliminary imaging frequency range until the imaging frequency range of the target sound source is determined.

[0060] Optionally, the newly added imaging frequency range is specifically used for:

[0061] Based on the spectrum, at least one undetermined frequency band whose sound source energy value exceeds the second screening threshold is determined;

[0062] Among at least one specific frequency band that exceeds the second screening threshold, after removing at least one specific frequency band that exceeds the first screening threshold, the remaining specific frequency bands are the new imaging frequency range of the target sound source.

[0063] An acoustic imaging frequency determination device includes a memory and a processor;

[0064] The memory is used to store programs;

[0065] The processor is used to execute the program to implement the various steps of the acoustic imaging frequency determination method as described above.

[0066] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the acoustic imaging frequency determination method as described above.

[0067] By employing the above technical solutions, this application discloses an acoustic imaging frequency determination method, apparatus, device, and readable storage medium. Generally, the possible location of a target sound source is specific. Therefore, in this application, a region of interest (ROI) is first preset based on the possible location of the target sound source. This ROI is a region in the video image to be acoustically imaged, and its area is smaller than the area of ​​the video image. Then, based only on the audio data corresponding to the preset ROI, the preliminary imaging frequency range of the target sound source is determined, which can reduce computational workload and power consumption, and improve frequency determination efficiency. After determining the preliminary imaging frequency range of the target sound source, an acoustic imaging map is generated for each preliminary imaging frequency range, combined with the video image. This map includes a sound field heat map corresponding to each preliminary imaging frequency range, the video image, and the ROI. Based on the positional relationship between the sound field heat map corresponding to each preliminary imaging frequency range and the ROI, the effective imaging frequency range of the target sound source is determined, which enables the determined frequency range of the target sound source to have high accuracy. Attached Figure Description

[0068] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0069] Figure 1 This is a flowchart illustrating an acoustic imaging frequency determination method disclosed in an embodiment of this application;

[0070] Figure 2 This is a schematic diagram of a region of interest disclosed in an embodiment of this application;

[0071] Figure 3This is a schematic diagram of a spectrum diagram disclosed in an embodiment of this application;

[0072] Figure 4 This is a schematic diagram of a sound field thermogram disclosed in an embodiment of this application;

[0073] Figure 5 This is a schematic diagram of a spectrum diagram disclosed in an embodiment of this application;

[0074] Figure 6 This is a schematic diagram of the structure of an acoustic imaging frequency determination device disclosed in an embodiment of this application;

[0075] Figure 7 This is a hardware structure block diagram of an acoustic imaging frequency determination device disclosed in an embodiment of this application. Detailed Implementation

[0076] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0077] Before introducing the proposed solution, some techniques for acoustic imaging will be introduced first.

[0078] Acoustic cameras (also known as acoustic imagers) use digital microphone arrays to visualize the collected sound as a color contour map (hereinafter referred to as a sound field heatmap) on a screen, creating an effect similar to how thermal imagers detect the temperature of objects. Acoustic cameras typically display video footage and sound field heatmaps in real time at a fixed frame rate. That is, they display the latest video frame and the sound field heatmap calculated based on the latest audio data frame (the duration of the audio data frame is fixed, such as 8ms).

[0079] The key to acoustic imaging lies in the overlay and display of video footage and sound field heatmaps. Video is defined by a frame rate, such as 25fps, meaning 25 frames can be captured per second. Similarly, audio is processed in frames, for example, 8ms per frame, resulting in 125 frames per second. Each frame of audio data is processed by algorithms to obtain corresponding sound field distribution data, i.e., sound intensity information at each point in space. This sound field distribution data is then transformed using coordinates to generate a sound field heatmap.

[0080] The spatial division is determined by arrays, algorithms, etc. For example, the space might be divided at 1° intervals in both the horizontal and vertical angles, ultimately resulting in m*n points. Sound field distribution data refers to the distribution of sound intensity across these m*n beams. Finally, each point in the space is mapped onto the camera image, meaning the video image and the sound field heatmap are superimposed and displayed on the screen.

[0081] Based on this, embodiments of this application provide an acoustic imaging frequency determination scheme, which can be applied to acoustic cameras. The acoustic imaging frequency determination method provided in this application will be described below through the following embodiments.

[0082] Reference Figure 1 , Figure 1 This is a flowchart illustrating an acoustic imaging frequency determination method disclosed in an embodiment of this application. The method may include:

[0083] Step S101: Based on the audio data corresponding to the preset region of interest, determine the preliminary imaging frequency range of the target sound source. The region of interest is a region in the video image to be acoustically imaged, and the area of ​​the region of interest is smaller than the area of ​​the video image.

[0084] Generally, the possible locations of target sound sources are specific. Therefore, in this application, a region of interest (ROI) is first preset based on the possible locations of the target sound source. In this application, the ROI is a region in the video image to be acoustically imaged, and the area of ​​the ROI is smaller than the area of ​​the video image. The video image to be acoustically imaged typically fills the entire screen area, and the ROI is a portion of that screen area. Specifically, the ROI can be determined by the user drawing a frame on the screen of the acoustic camera. The ROI can be a regular shape such as a rectangle, circle, or triangle, or it can be an irregular shape; this application does not impose any limitations on this. It should be noted that the user can adjust the border size of the ROI by adjusting the parameters of the acoustic camera according to the scene requirements, thereby adjusting the ROI.

[0085] For ease of understanding, please refer to Figure 2 , Figure 2 This is a schematic diagram of a region of interest disclosed in an embodiment of this application. Figure 2 As shown, the region of interest is a portion of the screen area.

[0086] It should be noted that the audio data corresponding to the preset region of interest is acquired by the digital microphone array in the acoustic camera. It can be understood that the digital acoustic microphones on the acoustic camera can be in the form of a microphone array, that is, containing multiple microphones. Therefore, the audio data corresponding to the preset region of interest can be multi-channel audio data, that is, it includes audio data acquired separately by each microphone.

[0087] In this application, the acoustic energy value of each preset frequency point can be calculated based on the audio data corresponding to the preset region of interest, and then the acoustic energy value of each preset frequency point can be filtered to obtain each preliminary imaging frequency range. The specific implementation will be described in detail through the following embodiments, and will not be described here.

[0088] Step S102: For each of the preliminary imaging frequency ranges, an acoustic imaging map is generated by combining the video image. The acoustic imaging map includes a sound field heat map corresponding to each of the preliminary imaging frequency ranges, the video image, and the region of interest.

[0089] In this application, acoustic field heatmaps for each of the preliminary imaging frequency ranges can be generated first, and then superimposed with the video image and the region of interest to obtain an acoustic imaging map.

[0090] Step S103: Based on the positional relationship between the acoustic field heatmaps corresponding to each of the preliminary imaging frequency ranges and the region of interest, determine the effective imaging frequency range of the target sound source.

[0091] In this application, for each of the preliminary imaging frequency ranges corresponding to the sound field heatmap, it can be determined whether the sound field heatmap corresponding to the preliminary imaging frequency range is in the region of interest, and a determination result is obtained. Based on the determination result, the preliminary imaging frequency range corresponding to the sound field heatmap in the region of interest is determined to be the effective imaging frequency range of the target sound source; the preliminary imaging frequency range corresponding to the sound field heatmap outside the region of interest is determined to be the invalid imaging frequency range.

[0092] The initial imaging frequency range of the target sound source can be determined in whole or in part as the effective imaging frequency range of the target sound source.

[0093] This embodiment discloses an acoustic imaging frequency determination method. Generally, the possible location of a target sound source is specific. Therefore, in this application, a region of interest (ROI) is first preset based on the possible location of the target sound source. The ROI is a region in the video image to be acoustically imaged, and its area is smaller than the area of ​​the video image. Then, based only on the audio data corresponding to the preset ROI, the preliminary imaging frequency range of the target sound source is determined, which can reduce the computational workload and power consumption and improve the efficiency of frequency determination. After determining the preliminary imaging frequency range of the target sound source, an acoustic imaging map is generated for each preliminary imaging frequency range, combined with the video image. This map includes a sound field heat map corresponding to each preliminary imaging frequency range, the video image, and the ROI. Based on the positional relationship between the sound field heat map corresponding to each preliminary imaging frequency range and the ROI, the effective imaging frequency range of the target sound source is determined, which enables the determined frequency range of the target sound source to have high accuracy.

[0094] In another embodiment of this application, the specific implementation of step S101, which determines the preliminary imaging frequency range of the target sound source based on the audio data corresponding to the preset region of interest, is described in detail. This method may include:

[0095] Step S201: Perform a fast Fourier transform on the audio data corresponding to the preset region of interest to obtain a spectrogram, which is used to indicate the sound source energy value at each preset frequency point.

[0096] In this application, a fast Fourier transform can be performed on the audio data corresponding to the preset region of interest to obtain a sound intensity matrix. Then, the sound source energy value of each preset frequency point can be determined based on the sound intensity matrix, and the spectrum can be determined based on the sound source energy value of each preset frequency point.

[0097] The preset frequency points can be determined as follows:

[0098] The set full frequency band is divided according to the target frequency interval to obtain a preset number of frequency points. For example, each frequency point contains a range of 60Hz, so 60kHz contains 1000 frequency points.

[0099] Step S202: Determine the first screening threshold based on the sound source energy value at each preset frequency point.

[0100] In one optional implementation, the process of determining the first screening threshold may include:

[0101] The sound source energy values ​​at each preset frequency point are averaged, and the result is used as the first screening threshold to characterize the lower limit of the target sound source energy.

[0102] In another alternative implementation, the process of determining the first screening threshold may include:

[0103] The energy values ​​of the sound sources at each preset frequency point are weighted and averaged, and the result is used as the first screening threshold to characterize the lower limit of the target sound source energy.

[0104] The weighting coefficients for each frequency point are preset values. For example, if the approximate frequency range of the target sound source is determined in advance, such as if the target sound source is determined to be in the low frequency range, then the weighting coefficients of each frequency point within the approximate frequency range of the target sound source can be increased, and the weighting coefficients of other frequency points can be decreased.

[0105] Step S202: Based on the spectrogram and the first screening threshold, determine the preliminary imaging frequency range of the target sound source.

[0106] As one possible implementation, determining the preliminary imaging frequency range of the target sound source based on the spectrogram and the first screening threshold includes:

[0107] S1. Based on the spectrum, determine at least one undetermined frequency band where the sound source energy value exceeds the first screening threshold.

[0108] Specifically, in the spectrum diagram, a straight line parallel to the horizontal frequency axis can be drawn, starting from the first screening threshold on the vertical axis. The spectrum waveform will then intersect this line. Waveforms above the line are selected, and the corresponding horizontal frequency bands can be used as the undetermined frequency bands.

[0109] For ease of understanding, please refer to Figure 3 , Figure 3 This is a schematic diagram of a spectrum diagram disclosed in an embodiment of this application. Figure 3 As shown:

[0110] In the spectrum graph, a straight line parallel to the horizontal axis is drawn at the first screening threshold, intersecting the spectrum waveform. The waveform above the line has four peaks, corresponding to frequency points i, i+1, j, and j+2. The frequency interval corresponding to the two intersection points of the waveform and the line at each peak is considered as a frequency band to be determined. Figure 3 The example has four undetermined frequency bands: 1, 2, 3, and 4.

[0111] S2. Based on the at least one undetermined frequency band, determine the preliminary imaging frequency range of the target sound source.

[0112] Specifically, in one optional approach, each undetermined frequency band can be used as the initial imaging frequency range of the target sound source.

[0113] In another alternative approach, the top N frequency bands with the largest maximum sound source energy values ​​can be selected as the initial imaging frequency range for the target sound source, based on the maximum sound source energy values ​​corresponding to each undetermined frequency band.

[0114] by Figure 3 Let's take an example: If N is defined as 2, then from... Figure 3 It can be seen that among the four undetermined frequency bands 1, 2, 3, and 4, the two with the largest maximum sound source energy are 1 and 4. Therefore, these two undetermined frequency bands, 1 and 4, can be used as the initial imaging frequency range for the target sound source.

[0115] Based on the above embodiments, after determining the effective imaging frequency range of the target sound source based on the positional relationship between the sound field heatmap corresponding to each of the preliminary imaging frequency ranges and the region of interest, the present application can further add a process of recommending the effective imaging frequency range of the target sound source to the user.

[0116] In another embodiment of this application, the process of recommending the effective imaging frequency range of the target sound source to the user is described as follows:

[0117] One possible approach is to recommend the entire effective imaging frequency range of the target sound source to the user. However, in some cases, the same sound source may involve multiple similar frequency ranges, and these similar frequency ranges are not significantly different. If all of them are recommended to the user, it will give the user too many options to filter, resulting in a poor subjective user experience.

[0118] To address the aforementioned issues, as another implementation method, the process of recommending the effective imaging frequency range of the target sound source to the user may include:

[0119] Step S201: Based on the acoustic field thermograms corresponding to each of the preliminary imaging frequency ranges, determine the recommended imaging frequency range from the effective imaging frequency range.

[0120] As one possible implementation, determining the recommended imaging frequency range from the effective imaging frequency range based on the acoustic field heatmaps corresponding to each of the preliminary imaging frequency ranges includes: calculating the distance between every two adjacent acoustic field heatmaps; if the distance between two adjacent acoustic field heatmaps is less than a preset distance threshold, obtaining the sound source energy value of the effective imaging frequency range corresponding to the two adjacent acoustic field heatmaps; determining the effective imaging frequency range with the larger sound source energy value among the effective imaging frequency ranges corresponding to the two adjacent acoustic field heatmaps as the recommended imaging frequency range; if the distance between two adjacent acoustic field heatmaps is greater than a preset distance threshold, determining the effective imaging frequency range corresponding to the two adjacent acoustic field heatmaps as the recommended imaging frequency range.

[0121] It should be noted that the distance between two adjacent sound field heatmaps can be the distance between the center points of the two adjacent sound field heatmaps.

[0122] Step S202: Calculate the distance between the acoustic field heatmap corresponding to each of the recommended imaging frequency ranges and the region of interest.

[0123] It should be noted that the distance between the acoustic field heatmap corresponding to each recommended imaging frequency range and the region of interest can be the distance between the center point of the acoustic field heatmap corresponding to the recommended imaging frequency range and the region of interest.

[0124] Step S203: According to the rule that the closer the distance, the higher the priority, sort the priority of each recommended imaging frequency range based on the distance between the acoustic field heat map corresponding to each recommended imaging frequency range and the region of interest, and obtain the sorted recommended imaging frequency ranges.

[0125] For ease of understanding, please refer to Figure 4 , Figure 4 This is a schematic diagram of a sound field thermogram disclosed in an embodiment of this application. Figure 4 As shown:

[0126] The acoustic field thermograms for the four undetermined frequency bands (1, 2, 3, and 4) of the initial imaging frequency range of the target sound source are as follows: Figure 1 ,heat Figure 2 ,heat Figure 3 and heat Figure 4 Among them, heat Figure 1 ,heat Figure 2 and heat Figure 3 Within the region of interest, thermal Figure 4 Outside the region of interest, the effective imaging frequency range of the target sound source is the three undetermined frequency bands 1, 2, and 3, which are all within the recommended imaging frequency range. (Thermal...) Figure 1 The closest to the center, followed by thermal. Figure 2 The furthest point is the heat map, so the sorted range of recommended imaging frequencies is as follows: 1, 2, 3.

[0127] Step S204: Recommend the sorted imaging frequency range to the user.

[0128] After obtaining the sorted frequency ranges to be recommended for imaging, they can be presented to the user. The user can then confirm whether each recommended frequency range matches the target sound source's imaging frequency range in the sorted order. Since higher priority frequency ranges have a higher probability of being identified as the target sound source's imaging frequency range, this helps the user determine the target sound source's imaging frequency range as quickly as possible.

[0129] In another embodiment of this application, after the sorted recommended imaging frequency ranges are recommended to the user, if the user does not determine the target sound source imaging frequency range from the sorted recommended imaging frequency ranges, the user can generate instruction information for re-filtering frequency bands by clicking the "Other Frequency Bands" option in the control terminal of the acoustic camera, and send the instruction information for re-filtering frequency bands to the acoustic camera. After receiving the instruction information for re-filtering frequency bands, the method further includes:

[0130] Step S301: Determine a second screening threshold, wherein the second screening threshold is less than the first screening threshold.

[0131] As one possible implementation, the second filtering threshold can be set to half of the first filtering threshold.

[0132] Step S302: Based on the spectrogram and the second screening threshold, determine the new imaging frequency range of the target sound source.

[0133] As one possible implementation, determining the new imaging frequency range of the target sound source based on the spectrum and the second screening threshold includes: determining at least one undetermined frequency band whose sound source energy value exceeds the second screening threshold based on the spectrum; and determining that among the at least one specific frequency band exceeding the second screening threshold, the remaining specific frequency band after removing the at least one specific frequency band exceeding the first screening threshold is the new imaging frequency range of the target sound source.

[0134] For ease of understanding, please refer to Figure 5 , Figure 5 This is a schematic diagram of a spectrum diagram disclosed in an embodiment of this application. Figure 5 As shown:

[0135] In the spectrum graph, a straight line parallel to the horizontal axis is drawn at the second screening threshold, intersecting the spectrum waveform. The waveform above the line has five peaks, corresponding to frequency points i, i+1, j, j+2, and j+4. The frequency interval corresponding to the two intersection points of the waveform and the line at each peak is considered as a frequency band to be determined. Figure 5 The example has five undetermined frequency bands: 1, 2, 3, 4, and 5. Among them, four undetermined frequency bands, 1, 2, 3, and 4, exceed the first screening threshold. Therefore, the newly added imaging frequency range is frequency band 5.

[0136] Step S303: After determining the new imaging frequency range, the new imaging frequency range is used as the new preliminary imaging frequency range for processing, and the acoustic imaging frequency determination method of the above embodiment is executed again until the imaging frequency range of the target sound source is determined.

[0137] Based on actual scenario testing, the energy value of the target frequency is generally greater than the background noise. The imaging frequency range of the target sound source can usually be determined based on the initial imaging frequency range determined in the first instance. In very few cases, a second determination of the initial imaging frequency range is required.

[0138] The acoustic imaging frequency determination apparatus disclosed in the embodiments of this application is described below. The acoustic imaging frequency determination apparatus described below and the acoustic imaging frequency determination method described above can be referred to in correspondence.

[0139] Reference Figure 6 , Figure 6 This is a schematic diagram of the structure of an acoustic imaging frequency determination device disclosed in an embodiment of this application. Figure 6 As shown, the acoustic imaging frequency determination device may include:

[0140] The preliminary imaging frequency range determination unit 11 is used to determine the preliminary imaging frequency range of the target sound source based on the audio data corresponding to the preset region of interest; the region of interest is a region in the video image to be acoustically imaged, and the area of ​​the region of interest is smaller than the area of ​​the video image.

[0141] The acoustic imaging map determination unit 12 is used to generate an acoustic imaging map for each of the preliminary imaging frequency ranges, in combination with the video image. The acoustic imaging map includes a sound field heat map corresponding to each of the preliminary imaging frequency ranges, the video image, and the region of interest.

[0142] The effective imaging frequency range determination unit 13 is used to determine the effective imaging frequency range of the target sound source based on the positional relationship between the sound field heat map corresponding to each of the preliminary imaging frequency ranges and the region of interest.

[0143] As one possible implementation, the preliminary imaging frequency range determination unit includes:

[0144] The spectrum determination subunit is used to perform a fast Fourier transform on the audio data corresponding to the preset region of interest to obtain a spectrum, which is used to indicate the sound source energy value at each preset frequency point;

[0145] The first screening threshold determination subunit is used to determine the first screening threshold based on the sound source energy value at each preset frequency point;

[0146] The preliminary imaging frequency range determination subunit is used to determine the preliminary imaging frequency range of the target sound source based on the spectrum and the first screening threshold.

[0147] As one possible implementation, the effective imaging frequency range determination unit is specifically used for:

[0148] The preliminary imaging frequency range corresponding to the sound field heatmap within the region of interest is determined as the effective imaging frequency range of the target sound source.

[0149] The preliminary imaging frequency range corresponding to the acoustic field thermogram outside the region of interest is determined to be the invalid imaging frequency range.

[0150] As one possible implementation, the device further includes:

[0151] The unit for determining the recommended imaging frequency range is used to determine the recommended imaging frequency range from the effective imaging frequency range based on the sound field heat map corresponding to each of the preliminary imaging frequency ranges and the positional relationship between the sound field heat map and the region of interest, after determining the effective imaging frequency range of the target sound source.

[0152] The distance calculation unit is used to calculate the distance between the acoustic field heatmap corresponding to each of the recommended imaging frequency ranges and the region of interest.

[0153] The sorting unit is used to sort the priority of each of the recommended imaging frequency ranges according to the rule that the closer the distance, the higher the priority, based on the distance between the acoustic field heat map corresponding to each of the recommended imaging frequency ranges and the region of interest, so as to obtain the sorted imaging frequency ranges to be recommended.

[0154] The recommendation unit is used to recommend the sorted imaging frequency range to the user.

[0155] As one possible implementation, the unit for determining the recommended imaging frequency range is specifically used for:

[0156] Calculate the distance between every two adjacent acoustic field heatmaps;

[0157] If the distance between two adjacent sound field heatmaps is less than a preset distance threshold, then the sound source energy value of the effective imaging frequency range corresponding to the two adjacent sound field heatmaps is obtained.

[0158] Among the effective imaging frequency ranges corresponding to two adjacent sound field heatmaps, the effective imaging frequency range with the larger sound source energy value is determined as the recommended imaging frequency range.

[0159] If the distance between two adjacent acoustic field heatmaps is greater than a preset distance threshold, then the effective imaging frequency range corresponding to the two adjacent acoustic field heatmaps is determined as the recommended imaging frequency range.

[0160] As one possible implementation, after recommending the sorted range of imaging frequencies to the user, if an instruction to further filter frequency bands is received; the device further includes:

[0161] The second screening threshold determination unit is used to determine a second screening threshold, wherein the second screening threshold is less than the first screening threshold;

[0162] A new imaging frequency range determination unit is added, which is used to determine the new imaging frequency range of the target sound source based on the spectrum and the second screening threshold.

[0163] The processing unit is used to process the newly added imaging frequency range as a new preliminary imaging frequency range until the imaging frequency range of the target sound source is determined.

[0164] As one possible implementation, the additional imaging frequency range is specifically used for:

[0165] Based on the spectrum, at least one undetermined frequency band whose sound source energy value exceeds the second screening threshold is determined;

[0166] Among at least one specific frequency band that exceeds the second screening threshold, after removing at least one specific frequency band that exceeds the first screening threshold, the remaining specific frequency bands are the new imaging frequency range of the target sound source.

[0167] Reference Figure 7 , Figure 7 A hardware structure block diagram of an acoustic imaging frequency determination device provided in this application embodiment is shown below. Figure 7 The hardware structure of an acoustic imaging frequency determination device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4.

[0168] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;

[0169] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0170] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0171] The memory stores a program, which the processor can call. The program is used for:

[0172] Based on the audio data corresponding to the preset region of interest, the preliminary imaging frequency range of the target sound source is determined; the region of interest is a region in the video image to be acoustically imaged, and the area of ​​the region of interest is smaller than the area of ​​the video image.

[0173] For each of the aforementioned preliminary imaging frequency ranges, an acoustic imaging map is generated by combining the video image. The acoustic imaging map includes a sound field heat map corresponding to each of the aforementioned preliminary imaging frequency ranges, the video image, and the region of interest.

[0174] Based on the positional relationship between the acoustic field heatmaps corresponding to each of the preliminary imaging frequency ranges and the region of interest, the effective imaging frequency range of the target sound source is determined.

[0175] Optionally, the refined and extended functions of the program can be found in the description above.

[0176] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:

[0177] Based on the audio data corresponding to the preset region of interest, the preliminary imaging frequency range of the target sound source is determined; the region of interest is a region in the video image to be acoustically imaged, and the area of ​​the region of interest is smaller than the area of ​​the video image.

[0178] For each of the aforementioned preliminary imaging frequency ranges, an acoustic imaging map is generated by combining the video image. The acoustic imaging map includes a sound field heat map corresponding to each of the aforementioned preliminary imaging frequency ranges, the video image, and the region of interest.

[0179] Based on the positional relationship between the acoustic field heatmaps corresponding to each of the preliminary imaging frequency ranges and the region of interest, the effective imaging frequency range of the target sound source is determined.

[0180] Optionally, the refined and extended functions of the program can be found in the description above.

[0181] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0182] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0183] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for determining acoustic imaging frequency, characterized in that, The method includes: Based on the audio data corresponding to the preset region of interest, the preliminary imaging frequency range of the target sound source is determined; the region of interest is a region in the video image to be acoustically imaged, and the area of ​​the region of interest is smaller than the area of ​​the video image. For each of the aforementioned preliminary imaging frequency ranges, an acoustic imaging map is generated by combining the video image. The acoustic imaging map includes a sound field heat map corresponding to each of the aforementioned preliminary imaging frequency ranges, the video image, and the region of interest. Based on the positional relationship between the acoustic field heatmaps corresponding to each of the preliminary imaging frequency ranges and the region of interest, the effective imaging frequency range of the target sound source is determined. Based on the acoustic field thermograms corresponding to each of the preliminary imaging frequency ranges, the recommended imaging frequency range is determined from the effective imaging frequency range. Calculate the distance between the acoustic field heatmap corresponding to each of the recommended imaging frequency ranges and the region of interest; According to the rule that the closer the distance, the higher the priority, the priority of each of the recommended imaging frequency ranges is sorted based on the distance between the sound field heatmap corresponding to each of the recommended imaging frequency ranges and the region of interest, so as to obtain the sorted imaging frequency ranges to be recommended. The sorted range of imaging frequencies to be recommended is then presented to the user.

2. The method according to claim 1, characterized in that, The determination of the preliminary imaging frequency range of the target sound source based on the audio data corresponding to the preset region of interest includes: A fast Fourier transform is performed on the audio data corresponding to the preset region of interest to obtain a spectrogram, which is used to indicate the sound source energy value at each preset frequency point; The first screening threshold is determined based on the sound source energy value at each preset frequency point; Based on the spectrogram and the first screening threshold, the preliminary imaging frequency range of the target sound source is determined.

3. The method according to claim 1, characterized in that, The determination of the effective imaging frequency range of the target sound source based on the positional relationship between the acoustic field heatmaps corresponding to each of the preliminary imaging frequency ranges and the region of interest includes: The preliminary imaging frequency range corresponding to the sound field heatmap within the region of interest is determined as the effective imaging frequency range of the target sound source. The preliminary imaging frequency range corresponding to the acoustic field thermogram outside the region of interest is determined to be the invalid imaging frequency range.

4. The method according to claim 1, characterized in that, The step of determining the recommended imaging frequency range from the effective imaging frequency range based on the acoustic field thermograms corresponding to each of the preliminary imaging frequency ranges includes: Calculate the distance between every two adjacent acoustic field heatmaps; If the distance between two adjacent sound field heatmaps is less than a preset distance threshold, then the sound source energy value of the effective imaging frequency range corresponding to the two adjacent sound field heatmaps is obtained. Among the effective imaging frequency ranges corresponding to two adjacent sound field heatmaps, the effective imaging frequency range with the larger sound source energy value is determined as the recommended imaging frequency range. If the distance between two adjacent acoustic field heatmaps is greater than a preset distance threshold, then the effective imaging frequency range corresponding to the two adjacent acoustic field heatmaps is determined as the recommended imaging frequency range.

5. The method according to claim 2, characterized in that, After recommending the sorted imaging frequency ranges to the user, if an instruction to further filter frequency bands is received, the method further includes: Determine a second screening threshold, which is less than the first screening threshold; Based on the spectrogram and the second screening threshold, the new imaging frequency range of the target sound source is determined; The newly added imaging frequency range is used as a new preliminary imaging frequency range for processing until the imaging frequency range of the target sound source is determined.

6. The method according to claim 5, characterized in that, The step of determining the new imaging frequency range of the target sound source based on the spectrogram and the second screening threshold includes: Based on the spectrum, at least one undetermined frequency band whose sound source energy value exceeds the second screening threshold is determined; Among at least one specific frequency band that exceeds the second screening threshold, after removing at least one specific frequency band that exceeds the first screening threshold, the remaining specific frequency bands are the new imaging frequency range of the target sound source.

7. An acoustic imaging frequency determination device, characterized in that, The device includes: The preliminary imaging frequency range determination unit is used to determine the preliminary imaging frequency range of the target sound source based on the audio data corresponding to the preset region of interest; the region of interest is a region in the video image to be acoustically imaged, and the area of ​​the region of interest is smaller than the area of ​​the video image. An acoustic imaging map determination unit is used to generate an acoustic imaging map for each of the preliminary imaging frequency ranges, in combination with the video image. The acoustic imaging map includes a sound field heat map corresponding to each of the preliminary imaging frequency ranges, the video image, and the region of interest. An effective imaging frequency range determination unit is used to determine the effective imaging frequency range of the target sound source based on the positional relationship between the acoustic field heat map corresponding to each of the preliminary imaging frequency ranges and the region of interest. The unit for determining the recommended imaging frequency range is used to determine the recommended imaging frequency range from the effective imaging frequency range based on the sound field heat map corresponding to each of the preliminary imaging frequency ranges and the positional relationship between the sound field heat map and the region of interest, after determining the effective imaging frequency range of the target sound source. The distance calculation unit is used to calculate the distance between the acoustic field heatmap corresponding to each of the recommended imaging frequency ranges and the region of interest. The sorting unit is used to sort the priority of each of the recommended imaging frequency ranges according to the rule that the closer the distance, the higher the priority, based on the distance between the acoustic field heat map corresponding to each of the recommended imaging frequency ranges and the region of interest, so as to obtain the sorted imaging frequency ranges to be recommended. The recommendation unit is used to recommend the sorted imaging frequency range to the user.

8. An acoustic imaging frequency determination device, characterized in that, Including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the acoustic imaging frequency determination method as described in any one of claims 1 to 6.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements each step of the acoustic imaging frequency determination method as described in any one of claims 1 to 6.

Images