Power grid frequency reference signal network construction and audio positioning method and device

Abstract

The invention relates to the technical field of signal processing, in particular to a power grid frequency reference signal network construction and audio positioning method and device, and the method comprises the steps: obtaining down-sampled audio from multi-region online real-time multimedia data; when detecting that the audio has the power grid frequency, processing the audio to obtain a power grid frequency signal of a single audio; processing the power grid frequency signals of the multiple audios in the same region to obtain power grid frequency reference signals of a single region, and processing the power grid frequency reference signals of the single region in multiple regions to obtain a power grid frequency reference signal network; and processing the to-be-positioned audio to obtain a to-be-positioned audio power grid frequency signal, and calculating a correlation coefficient between the to-be-positioned power grid frequency signal and each network point of the power grid frequency reference signal network to obtain a final positioning result of the audio. According to the invention, power grid frequency reference signal network construction and audio positioning can be realized based on online real-time multimedia data.

Description

Technical Field

[0001] The present application relates to the field of signal processing technology, and in particular to a method and device for constructing a power grid frequency reference signal network and audio positioning. Background Art

[0002] With the rapid development of science and technology, audio storage and transmission technology is also constantly improving. In this context, the audio data in our daily lives is experiencing exponential growth. However, when the audio is recorded, the information of its original recording location is often not recorded. In application scenarios that strictly rely on recording location information, such as security monitoring, it is particularly important to be able to accurately determine the recording location of the audio.

[0003] In the related technology, the traditional audio positioning method uses the power grid frequency for audio positioning based on the construction of a power grid frequency reference signal network, which usually requires the deployment of a large number of frequency interference recorders at different locations. With the continuous development of network transmission technology, a large amount of multimedia content can now be transmitted in real time on the network, providing a new way to obtain data. These real-time multimedia data not only contain rich geographic location information, but also naturally embed power grid frequency signals, providing a new type of power grid frequency reference signal network data source for audio positioning.

[0004] However, traditional audio positioning methods in related technologies require the deployment of a large number of frequency interference recorders, which are costly and complex to maintain. They also face harsh usage conditions, poor positioning accuracy, and low positioning accuracy. Their application scope is not wide enough and cannot meet the needs of audio positioning in practical applications, which needs to be solved urgently. Summary of the invention

[0005] The present application provides a method and device for constructing a power grid frequency reference signal network and audio positioning, so as to solve the problems that the traditional audio positioning method in the related technology needs to deploy a large number of frequency interference recorders, which is costly and complex to maintain, and also faces the problems of harsh use conditions, poor positioning precision, low positioning accuracy, and a limited application scope, and cannot meet the needs of audio positioning in practical applications.

[0006] The first aspect of the present application provides a method for constructing a power grid frequency reference signal network and positioning an audio, comprising the following steps: extracting original audio from online real-time multimedia data in multiple regions, and downsampling the original audio to obtain the downsampled audio; detecting whether the audio has a power grid frequency, and processing the audio to obtain a power grid frequency signal of a single audio when it is detected that the audio has the power grid frequency; processing the power grid frequency signals of multiple audios in the same region to obtain a single-region power grid frequency reference signal, and processing the single-region power grid frequency reference signals in multiple regions to obtain a power grid frequency reference signal network; processing the audio to be located to obtain the power grid frequency signal of the audio to be located, and calculating the correlation coefficient between the power grid frequency signal to be located and each grid point of the power grid frequency reference signal network to obtain the final positioning result of the audio according to the correlation coefficient.

[0007] Optionally, in one embodiment of the present application, detecting whether the audio has a grid frequency includes: obtaining a signal length of a grid frequency signal of the audio; and calculating a noise segment rate based on the signal length to determine whether the audio has a grid frequency based on the noise segment rate.

[0008] Optionally, in one embodiment of the present application, the processing of the audio to obtain a single audio grid frequency signal includes: enhancing the grid frequency signal of the audio to obtain an enhanced audio signal; extracting a multi-harmonic grid frequency signal of the enhanced audio signal; and selecting and fusing the multi-harmonic grid frequency signal to obtain the single audio grid frequency signal.

[0009] Optionally, in one embodiment of the present application, the grid frequency signals of multiple audio frequencies in the same area are processed to obtain a single-area grid frequency reference signal, including: constructing a target spatial domain selection matrix based on the signal length of the grid frequency signal; converting the target spatial domain selection matrix into a target spatial domain selection undirected graph, and fusing the spatial domain signals of the target spatial domain selection undirected graph to obtain the single-area grid frequency reference signal.

[0010] Optionally, in one embodiment of the present application, the processing of the single-region power grid frequency reference signal of multiple regions to obtain a power grid frequency reference signal network includes: dividing the total area to be located in the multiple regions into multiple grids, and obtaining the grid point coordinates of the multiple grids; collecting the center point coordinates of the single-region power grid frequency reference signal of the multiple regions; calculating the distance between the grid point coordinates of the multiple grids and the center point coordinates, so as to obtain the power grid frequency reference signal network based on the distance.

[0011] The second aspect of the present application provides a power grid frequency reference signal network construction and audio positioning device, including: an extraction module, used to extract original audio from online real-time multimedia data in multiple regions, and downsample the original audio to obtain the downsampled audio; a first processing module, used to detect whether the audio has a power grid frequency, and when it is detected that the audio has the power grid frequency, process the audio to obtain a power grid frequency signal of a single audio; a second processing module, used to process the power grid frequency signals of multiple audios in the same region to obtain a single-region power grid frequency reference signal, and process the single-region power grid frequency reference signals in multiple regions to obtain a power grid frequency reference signal network; a positioning module, used to process the audio to be positioned to obtain the power grid frequency signal of the audio to be positioned, and calculate the correlation coefficient between the power grid frequency signal to be positioned and each network point of the power grid frequency reference signal network, so as to obtain the final positioning result of the audio according to the correlation coefficient.

[0012] Optionally, in one embodiment of the present application, the first processing module includes: a first acquisition unit, used to acquire the signal length of the power grid frequency signal of the audio; and a determination unit, used to calculate the noise segment rate based on the signal length, so as to determine whether the audio has a power grid frequency based on the noise segment rate.

[0013] Optionally, in one embodiment of the present application, the first processing module includes: an enhancement unit, used to enhance the power grid frequency signal of the audio to obtain an enhanced audio signal; an extraction unit, used to extract the multi-harmonic power grid frequency signal of the enhanced audio signal; a first processing unit, used to select and fuse the multi-harmonic power grid frequency signal to obtain the single audio power grid frequency signal.

[0014] Optionally, in one embodiment of the present application, the second processing module includes: a construction unit, used to construct a target spatial domain selection matrix based on the signal length of the power grid frequency signal; a second processing unit, used to convert the target spatial domain selection matrix into a target spatial domain selection undirected graph, and fuse the spatial domain signals of the target spatial domain selection undirected graph to obtain the single area power grid frequency reference signal.

[0015] Optionally, in one embodiment of the present application, the second processing module includes: a second acquisition unit, used to divide the total area to be positioned in the multiple regions into multiple grids, and obtain the grid point coordinates of the multiple grids; a collection unit, used to collect the center point coordinates of the single-region power grid frequency reference signal of the multiple regions; a calculation unit, used to calculate the distance between the grid point coordinates of the multiple grids and the center point coordinates, so as to obtain the power grid frequency reference signal network based on the distance.

[0016] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the grid frequency reference signal network construction and audio positioning method as described in the above embodiments.

[0017] The fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the program is executed by a processor, it implements the above-mentioned power grid frequency reference signal network construction and audio positioning method.

[0018] The fifth aspect of the present application provides a computer program product, including a computer program, which, when executed, is used to implement the above-mentioned power grid frequency reference signal network construction and audio positioning method.

[0019] The embodiment of the present application can use online real-time multimedia data to construct a power grid frequency reference signal network, and use the reference network to locate the audio data, thereby realizing the use of online real-time multimedia data as the source of the power grid frequency reference signal, avoiding the construction cost and maintenance cost of the power grid frequency reference network, while taking advantage of the large number of online multimedia data, which can be collected at multiple points, effectively ensuring the geographical resolution of power grid frequency signal collection, and improving the accuracy of the power grid frequency reference signal network. Furthermore, by using the power grid frequency reference network to locate the audio, the efficiency of audio positioning in the present application is effectively improved, the scope of positioning application is expanded, and it is convenient for actual implementation. Thus, the problem that the traditional audio positioning method in the related art needs to deploy a large number of frequency interference recorders, which is costly and complex to maintain, and also faces the problems of harsh use conditions, poor positioning accuracy, low positioning accuracy, insufficient application scope, and inability to meet the needs of audio positioning in actual applications is solved.

[0020] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of a method for constructing a power grid frequency reference signal network and audio positioning according to an embodiment of the present application;

[0023] Figure 2 This is a schematic diagram of the online audio power grid frequency signal existence judgment result according to one embodiment of the present application;

[0024] Figure 3 This is a schematic diagram of online audio power grid frequency signal enhancement and multi-harmonic extraction results according to an embodiment of the present application;

[0025] Figure 4 This is a schematic diagram of online audio power grid frequency harmonic selection and fusion results according to an embodiment of the present application;

[0026] Figure 5 A schematic diagram of grid frequency spatial domain selection and fusion results according to an embodiment of the present application;

[0027] Figure 6 This is a schematic diagram of an audio localization result according to an embodiment of the present application;

[0028] Figure 7 A flowchart of a method for constructing a power grid frequency reference signal network and audio positioning according to an embodiment of the present application;

[0029] Figure 8 A schematic diagram of the structure of a power grid frequency reference signal network construction and an audio positioning device according to an embodiment of the present application;

[0030] Fig. 9 Schematic diagram of the structure of an electronic device according to an embodiment of the present application.

[0031] Reference numerals:

[0032] 10-Grid frequency reference signal network construction and audio positioning device: 100-extraction module, 200-first processing module, 300-second processing module and 400-positioning module; 901-memory, 902-processor and 903-communication interface. DETAILED DESCRIPTION

[0033] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0034] The following describes the power grid frequency reference signal network construction and audio positioning method and device of the embodiment of the present application with reference to the accompanying drawings. In view of the fact that the traditional audio positioning method in the related art mentioned in the above background technology requires the deployment of a large number of frequency interference recorders, which is costly and complex to maintain, and also faces the problems of harsh use conditions, poor positioning accuracy, and low positioning accuracy, and the application scope is not wide enough, and it cannot meet the needs of audio positioning in practical applications, the present application provides a power grid frequency reference signal network construction and audio positioning method, in which the power grid frequency reference signal network can be constructed using online real-time multimedia data, and the reference network can be used to locate the audio data, thereby realizing the use of online real-time multimedia data as the source of the power grid frequency reference signal, avoiding the construction cost and maintenance cost of the power grid frequency reference network, and utilizing the characteristics of a large number of online multimedia data that can be collected at multiple points, effectively ensuring the geographical resolution of the power grid frequency signal collection, and improving the accuracy of the power grid frequency reference signal network, and further, by using the power grid frequency reference network to locate the audio, the efficiency of the audio positioning in the present application is effectively improved, the positioning application scope is expanded, and it is convenient for actual implementation. This solves the problem that the traditional audio positioning method in the related technology needs to deploy a large number of frequency interference recorders, which is costly and complex to maintain. It also faces the problems of harsh usage conditions, poor positioning accuracy, low positioning accuracy, and a limited application range, which cannot meet the needs of audio positioning in actual applications.

[0035] Specifically, Figure 1 A flowchart of a grid frequency reference signal network construction and audio positioning method provided in an embodiment of the present application.

[0036] like Figure 1 As shown, the grid frequency reference signal network construction and audio positioning method includes the following steps:

[0037] In step S101, original audio is extracted from multi-region online real-time multimedia data, and down-sampled to obtain down-sampled audio.

[0038] It is understandable that the present application can realize the construction of the power grid frequency reference signal network and audio positioning based on multi-region online real-time multimedia data. In some embodiments, before realizing this task, it is first necessary to extract the original audio from the multi-region online real-time multimedia data, and further, the original audio can be downsampled, which helps to use the downsampled audio to realize the construction of the power grid frequency reference signal network and audio positioning in the subsequent process.

[0039] For example, in the embodiment of the present application, the relevant total area to be located can be defined as P o , you can choose different regions Pi As an online real-time multimedia data collection area, P i ∈P o ,i=1,2…I, I represents the total number of online real-time multimedia data collection areas.

[0040] By searching the online real-time multimedia data platform, the online real-time multimedia data collection area P i The identified data source is downloaded and saved in real time to obtain a number of online real-time original audio data S in each region i_j , where S i_j Indicates that from region P i The jth audio collected, j = 1, 2...J, J represents the total number of audio collected in this area. Then, each original audio is downsampled to obtain the available downsampled audio data S i_j_down .

[0041] The embodiment of the present application can extract original audio from online real-time multimedia data in multiple regions and downsample the original audio. By using the audio data after downsampling, the storage space and transmission bandwidth requirements of the audio data can be reduced, and the speed of audio processing may be accelerated.

[0042] Step S102, detecting whether the audio has a power grid frequency, and in the case where it is detected that the audio has a power grid frequency, processing the audio to obtain a single audio power grid frequency signal.

[0043] It is understandable that the nominal value of the grid frequency (Electric Network Frequency, ENF), that is, the transmission frequency of alternating current, is usually determined by the region. Under normal circumstances, the nominal value of my country's grid frequency is 50Hz, and the nominal value of the grid frequency in the United States and other countries is 60Hz. Due to the influence of various factors such as power generation adjustment and changes in consumer demand, the actual grid frequency is not constant, but needs to be monitored and adjusted in real time by the power control departments of various regions to maintain stability. This frequency adjustment change is not instantaneous for the same power grid, but propagates within the power grid at a certain speed. Therefore, for the same power grid, the grid frequencies in different locations are slightly different. Based on this, the embodiment of the present application can first process the audio to obtain a grid frequency signal of a single audio.

[0044] In some embodiments, after extracting the original audio from the online real-time multimedia data and downsampling the original audio to obtain usable audio, the collected audio can be detected, that is, whether there is a power grid frequency in the audio, so that audio carrying the power grid frequency signal can be obtained.

[0045] Furthermore, the embodiments of the present application can also perform grid frequency signal enhancement, multi-harmonic extraction, selection and fusion on the audio carrying the grid frequency signal to obtain a single audio grid frequency signal.

[0046] Next, how to detect whether the audio has the power grid frequency and how to process the audio to obtain a single audio power grid frequency signal will be further described.

[0047] Optionally, in one embodiment of the present application, detecting whether the audio has a power grid frequency includes: acquiring a signal length of a power grid frequency signal of the audio; and calculating a noise segment rate based on the signal length to determine whether the audio has a power grid frequency based on the noise segment rate.

[0048] In some embodiments, when detecting whether there is a grid frequency in the audio, the present application can make a judgment based on the signal length of the grid frequency signal in the audio, such as calculating the noise fragment rate of the grid frequency signal in the audio by the signal length of the grid frequency signal in the audio. If the noise fragment rate meets certain conditions, it can be determined that there is a grid frequency in the audio.

[0049] For example, each downsampled audio data S i_j_down Perform short-time Fourier transform to obtain the time-frequency domain sequence X corresponding to the audio data i_j (k,n), where X i_j Indicates S i_j_down Wherein, k represents the kth frame of short-time Fourier transform (STFT), n represents the nth frequency point of STFT transform, and the total number of STFT frequency points can be set to N.

[0050] Next, use quadratic interpolation to obtain X i_j ENF multiple harmonics estimate E of (k,n) i_j_m [k], where E i_j_m Indicates audio data S i_j_down The ENF prediction value estimated by the mth subharmonic of , m∈M, M={2,3,4,5,6,7}.

[0051] Then, for E i_j_m [k] To judge the existence of the power grid frequency. Since the power grid frequency fluctuation range and fluctuation change rate are usually kept in a fixed range, we can set the following power grid frequency signal judgment criteria for each harmonic, which can be expressed as follows:

[0052] f down *m≤E i_j_m [k]≤f up *m,

[0053] Ei_j_m [k]-E i_j_m [k-1]≤ε r *m,

[0054] Among them, f up 、f down is the upper and lower limits of the grid frequency, ε r Indicates the maximum rate of change of the power grid frequency. For any E i_j_m For example, if the ENF value at a certain point does not meet the grid frequency signal criterion, then the point and the m on both sides of the point noise All points will be judged as noise signals.

[0055] Therefore, based on the above criteria, if a certain power grid frequency signal E i_j_m [k], the signal length is k o , the length of the signal judged as noise is k no , then the noise fragment rate η can be defined as:

[0056]

[0057] Generally speaking, for audio data S with ENF i_j_down , there is at least one harmonic whose noise fragment rate satisfies the condition and can be expressed as follows:

[0058] η<ε n ,

[0059] Among them, ε n is the noise control factor. In the embodiment of the present application, the noise control factor can be 50%. According to this judgment method, S i_j_down There is an audio signal S of the power grid frequency in i_h , S i_h Indicates that from region P i The h-th audio containing ENF signal collected, h=1,2…H, H represents the total number of audio containing ENF signal collected in this area.

[0060] Optionally, in one embodiment of the present application, the audio is processed to obtain a single audio grid frequency signal, including: enhancing the grid frequency signal of the audio to obtain an enhanced audio signal; extracting a multi-harmonic grid frequency signal of the enhanced audio signal; and selecting and fusing the multi-harmonic grid frequency signal to obtain a single audio grid frequency signal.

[0061] In the actual implementation process, in order to facilitate accurate positioning of the audio, the embodiment of the present application can process the audio to obtain a single audio grid frequency signal. When processing the audio, it can be, but is not limited to, performing grid frequency signal enhancement, multi-harmonic extraction, selection and fusion processing on the audio carrying the grid frequency signal to obtain a single audio grid frequency signal.

[0062] For example, the power grid frequency signal of the audio is first enhanced to obtain the enhanced audio signal, the multi-harmonic power grid frequency signal of the enhanced audio signal is extracted, and then the multi-harmonic power grid frequency signal is selected and fused to obtain a single audio power grid frequency signal.

[0063] For example, for each audio signal S with an ENF (Electric Network Frequency) signal i_h For example, first, the ENF signal can be enhanced by using a harmonic robust filtering algorithm (HRFA), but is not limited to, and after obtaining the enhanced audio signal, the multi-harmonic ENF signal is extracted by using the STFT and quadratic interpolation method to obtain the extracted multi-harmonic power grid frequency signal E i_h_m [k], where E i_h_m Indicates audio data S i_h The ENF prediction signal estimated by the mth subharmonic of the signal has a signal length of k i_h_o .

[0064] Next, the extracted E i_h_m [k] For harmonic selection, we can first construct an m×m order harmonic selection matrix through the following formula, which can be expressed as follows:

[0065]

[0066] Among them, R i_h_fre For audio data S i_h The harmonic selection matrix, {R i_h_fre} ab represents the element at the position (a, b) of the matrix, ρ(·,·) represents the calculation of the Pearson correlation coefficient between the two, E i_h_a [k], E i_h_b [k] represents audio data S i_h The ENF prediction signal estimated from the a-order subharmonic and the b-order subharmonic. fc is the frequency domain correlation coefficient control factor, which can be calculated by the following formula:

[0067] ε fc =min(κ a ,κ f εfn ),

[0068] Among them, min(·,·) means taking the minimum value between the two, κ a is the frequency domain minimum correlation coefficient control parameter, which can be 0.88 in the embodiment of the present application; κ f is the frequency domain coefficient magnification factor. In the embodiment of the present application, the frequency domain coefficient magnification factor can be 4; ε fn is the pseudo-random signal control factor, which can be calculated using the following formula:

[0069]

[0070] Among them, WGN(k i_h_o ) indicates that the generated length is k i_h_o Gaussian white noise sequence, ρ(·,·) represents the calculation of the Pearson correlation coefficient between the two. Indicates that the operation is repeated N times F times, N F is the number of Gaussian white noise frequency domain generation times. In the embodiment of the present application, the number of Gaussian white noise frequency domain generation times can be 1×10 5 , Indicates taking the N in the brackets F The maximum value of the number.

[0071] Then, the harmonic selection matrix R i_h_fre Transformed into a harmonic selection undirected graph G f , select through the maximum weight group algorithm in graph theory to obtain the available harmonic number M f Finally, the maximum weighted likelihood estimation method is used for fusion to obtain a single audio grid frequency signal E i_h [k], where E i_h Indicates audio data S i_h The final grid frequency estimation result.

[0072] Take a certain place in Central China Power Grid as an example. Figure 3 This is a graph showing the online audio grid frequency signal enhancement and multi-harmonic extraction results of an embodiment of the present application. Figure 4 for Figure 3 The harmonic selection and fusion result diagram of the multi-harmonic power grid frequency signal is shown in the figure, where the harmonic selection result is M f ={2,4,6,7}, the local reference signal is the power grid frequency signal directly recorded by the frequency interference recorder in the area. The results show that the power grid frequency signal E extracted from the online audio i_h The correlation coefficient between [k] and the local reference signal is 99.089%.

[0073] Step S103, processing the grid frequency signals of multiple audio frequencies in the same region to obtain a single-region grid frequency reference signal, and processing the single-region grid frequency reference signals of multiple regions to obtain a grid frequency reference signal network.

[0074] In other embodiments, after obtaining a single-frequency grid frequency signal, the present application can process grid frequency signals of multiple frequencies in the same region based on the single-frequency grid frequency signal to obtain a single-region grid frequency reference signal.

[0075] Furthermore, the embodiments of the present application may also use a single-region power grid frequency reference signal of multiple regions as an anchor point, process it, and thereby obtain a power grid frequency reference signal network.

[0076] Next, we will further explain how to process the power grid frequency signals of multiple audio frequencies in the same area to obtain a single-area power grid frequency reference signal and how to use a single-area power grid frequency reference signal in multiple regions as an anchor point to obtain a power grid frequency reference signal network.

[0077] Optionally, in one embodiment of the present application, the grid frequency signals of multiple audio frequencies in the same area are processed to obtain a single-area grid frequency reference signal, including: constructing a target spatial domain selection matrix based on the signal length of the grid frequency signal; converting the target spatial domain selection matrix into a target spatial domain selection undirected graph, and fusing the spatial domain signals of the target spatial domain selection undirected graph to obtain a single-area grid frequency reference signal.

[0078] In the actual implementation process, when processing the power grid frequency signals of multiple audio frequencies in the same region, the present application can, but is not limited to, perform spatial selection and fusion processing on the power grid frequency signals of multiple audio frequencies in the same region, thereby obtaining a single regional power grid frequency reference signal. Specifically, a target spatial selection matrix can be first constructed based on the signal length of the power grid frequency signal, and then the target spatial selection matrix can be converted into a target spatial selection undirected graph, and the spatial signals of the target spatial selection undirected graph can be fused, thereby obtaining a single regional power grid frequency reference signal.

[0079] For example, for the same region P i The extracted power grid frequency signal E i_h [k], the length of each signal is k i First, we can construct an H×H order spatial selection matrix by the following formula:

[0080]

[0081] Among them, R i_spa For region P i The spatial domain selection matrix, {R i_spa} abrepresents the element at the position (a, b) of the matrix, ρ(·,·) represents the calculation of the Pearson correlation coefficient between the two, E i_a [k], E i_b [k] indicates region P i Medium audio data S i_a and S i_b ENF estimation results, ε sc is the spatial correlation coefficient control factor, which can be calculated by the following formula:

[0082] ε sc =min(κ b ,κ s ε sn ),

[0083] Among them, min(·,·) means taking the minimum value between the two, κ b is the spatial minimum correlation coefficient control parameter, which can be 0.9 in the embodiment of the present application; κ s is the spatial coefficient magnification factor. In the embodiment of the present application, the spatial coefficient magnification factor can be 4; ε sn is the spatial pseudo-random signal control factor, which can be calculated using the following formula:

[0084]

[0085] Among them, WGN(k i ) indicates that the generated length is k i Gaussian white noise sequence, ρ(·,·) represents the calculation of the Pearson correlation coefficient between the two. Indicates that the operation is repeated N times S times, N S is the number of Gaussian white noise spatial domain generation times. In the embodiment of the present application, the number of Gaussian white noise spatial domain generation times can be 1×10 5 , Indicates taking the N in the brackets S The maximum value of the number.

[0086] Then, the spatial domain selection matrix R i_spa Transformed into a spatial selection undirected graph G s , select through the maximum weight group algorithm in graph theory to obtain the available spatial signal set H f Finally, the maximum weighted likelihood estimation method is used for fusion to obtain the frequency signal E of the power grid in various places. i [k], where E i Indicates region P i The final grid frequency reference signal estimation result.

[0087] Take a certain place in Central China Power Grid as an example. Figure 5 This is a diagram of the grid frequency spatial domain selection and fusion results of an embodiment of the present application. Figure 5 As shown, the local reference signal is the power grid frequency signal directly recorded by a frequency interference recorder in the area. The results show that the correlation coefficient between the ENF signal after spatial domain fusion and the local reference signal is 99.473%.

[0088] Optionally, in one embodiment of the present application, a single-region power grid frequency reference signal of multiple regions is processed to obtain a power grid frequency reference signal network, including: dividing the total area to be located in the multiple regions into multiple grids, and obtaining the grid point coordinates of the multiple grids; collecting the center point coordinates of the single-region power grid frequency reference signal of the multiple regions; calculating the distance between the grid point coordinates of the multiple grids and the center point coordinates to obtain the power grid frequency reference signal network based on the distance.

[0089] In certain embodiments, when processing a single-region power grid frequency reference signal from multiple regions to obtain a power grid frequency reference signal network, the multi-region power grid frequency reference signal can be used as an anchor point and the inverse distance weighted method can be used for geographic interpolation to obtain the power grid frequency reference signal network.

[0090] Specifically, the total area to be located in multiple regions can be first divided into multiple grids, and the grid point coordinates of these grids can be obtained; then the center point coordinates of the power grid frequency reference signal of a single region in multiple regions can be collected; finally, the power grid frequency reference signal network can be obtained by calculating the distance between the grid point coordinates of multiple grids and the center point coordinates.

[0091] For example, for the total area P to be located o For example, an interpolation grid of size W×W can be generated, and the coordinates of each grid point can be expressed as:

[0092] (x A ,y B ),A,B∈W,

[0093] At the same time, the power grid frequency signal area P is collected i The center point coordinates can be expressed as:

[0094] (x p_i ,y p_i ),i∈I,

[0095] Next, each point of the interpolation network can be interpolated using the following formula, but is not limited to:

[0096]

[0097] Among them, E A×B [k] indicates the dot coordinates (x A ,y B ) at the grid frequency signal, λ qis the inverse distance weight coefficient, which can be calculated using the following formula:

[0098]

[0099] Wherein, p is the inverse distance weighting coefficient, and in the embodiment of the present application, the inverse distance weighting coefficient can be 2; d q For each area P where the grid frequency signal is collected from the network point i The distance of the center point coordinates can be calculated using the following formula:

[0100]

[0101] Perform inverse distance weighted interpolation on each grid point in the interpolation grid to obtain the grid frequency reference signal network E W×W [k].

[0102] It should be noted that the parameter values ​​in the embodiments of the present application are for illustrative purposes only. Professionals and technicians in this field can make adjustments and changes according to specific circumstances in actual applications, and this application does not impose any specific limitations.

[0103] Step S104, processing the audio to be located to obtain the grid frequency signal of the audio to be located, and calculating the correlation coefficient between the grid frequency signal to be located and each grid point of the grid frequency reference signal network to obtain the final positioning result of the audio according to the correlation coefficient.

[0104] As a possible implementation method, after obtaining the grid frequency reference signal network, the embodiment of the present application can perform grid frequency signal enhancement, multi-harmonic extraction and fusion on the audio to be located to obtain the grid frequency signal of the audio to be located, and then calculate the correlation coefficient between the grid frequency signal to be located and each grid point of the grid frequency reference signal network respectively, and finally, according to the size of the correlation coefficient, the final positioning result of the audio can be obtained.

[0105] Specifically, for a certain audio signal S to be located O , it can be downsampled first to obtain the downsampled audio signal S O_down Next, for S O_down The HRFA ​​algorithm is used to enhance the ENF signal. After obtaining the enhanced audio signal, the multi-harmonic ENF signal is extracted using the STFT and quadratic interpolation method to obtain the extracted multi-harmonic power grid frequency signal E O_m [k], where E O_m Represents audio data W O_down The ENF prediction signal is estimated from the mth subharmonic, and then the multi-harmonic selection and fusion algorithm is used to obtain the audio data S O_down Final grid frequency signal E O [k].

[0106] To obtain the audio ENF signal E to be located O [k] can be calculated using the following formula, but is not limited to, the grid frequency reference signal network E W×W [k] Correlation coefficient between each point:

[0107] CC A×B =ρ(E A×B [k],E O [k]),

[0108] Among them, E A×B [k] represents the grid point coordinates (x A ,y B ), and ρ(·,·) represents the calculation of the Pearson correlation coefficient between the two.

[0109] After obtaining the correlation coefficient, the final positioning result can be obtained according to the following criterion:

[0110] CC A×B ≥ε Cc ,

[0111] Among them, ε CC It is the positioning correlation coefficient determination factor. The area containing all the points that meet this determination condition is the final audio positioning result.

[0112] Take a certain place in Central China Power Grid as an example. Figure 6 FIG. 1 is a schematic diagram of an audio localization result of an embodiment of the present application. Figure 6 As shown in the figure, the square area is the anchor point, the five-pointed star area is the audio recording location to be located, and the color filled in the map represents the audio grid frequency signal E O [k] and the grid frequency reference signal network E w×w [k] The correlation coefficient between each point. The smaller the correlation coefficient, the darker the color. In this example, the positioning correlation coefficient determination factor ε cc Take 97.3%, the positioning result is the closed area pointed by the arrow, the positioning result area contains the audio recording position to be located, and the positioning is successful.

[0113] The present application is described in detail below with reference to a specific embodiment.

[0114] Figure 7 This is a flow chart of a method for constructing a grid frequency reference signal network and audio positioning according to an embodiment of the present application. Figure 7 As shown:

[0115] Step 1: Collect online real-time multimedia data from multiple regions, extract its audio, and downsample the audio;

[0116] Step 2: Determine the existence of the power grid frequency on the collected audio to obtain the audio with the power grid frequency signal;

[0117] Step 3: Perform grid frequency signal enhancement, multi-harmonic extraction, selection and fusion on the audio carrying the grid frequency signal to obtain a single audio grid frequency signal;

[0118] Step 4: Perform spatial selection and fusion on the power grid frequency signals of multiple audio frequencies in the same area to obtain a single-area power grid frequency reference signal;

[0119] Step 5: Use the multi-regional power grid frequency reference signals as anchor points and use the inverse distance weighted method to perform geographic interpolation to obtain the power grid frequency reference signal network;

[0120] Step 6: Perform grid frequency signal enhancement, multi-harmonic extraction and fusion on the audio to be located to obtain the grid frequency signal of the audio to be located;

[0121] Step 7: Calculate the correlation coefficients between the grid frequency signal to be located and each grid point of the grid frequency reference signal network, and obtain the final positioning result based on the size of the correlation coefficient.

[0122] According to the grid frequency reference signal network construction and audio positioning method proposed in the embodiment of the present application, online real-time multimedia data can be used to construct a grid frequency reference signal network, and the reference network can be used to locate the audio data, thereby realizing the use of online real-time multimedia data as the source of the grid frequency reference signal, avoiding the construction cost and maintenance cost of the grid frequency reference network, while taking advantage of the large number of online multimedia data, which can be collected at multiple points, effectively ensuring the geographical resolution of grid frequency signal collection, and improving the accuracy of the grid frequency reference signal network. Furthermore, by using the grid frequency reference network to locate the audio, the efficiency of audio positioning in the present application is effectively improved, the scope of positioning application is expanded, and it is convenient for actual implementation. Thus, the problem that the traditional audio positioning method in the related art needs to deploy a large number of frequency interference recorders, which is costly and complex to maintain, and also faces the problems of harsh use conditions, poor positioning accuracy, low positioning accuracy, insufficient application scope, and inability to meet the needs of audio positioning in actual applications is solved.

[0123] Next, the power grid frequency reference signal network construction and audio positioning device proposed in accordance with the embodiments of the present application will be described with reference to the accompanying drawings.

[0124] Figure 8 It is a structural diagram of a power grid frequency reference signal network construction and an audio positioning device according to an embodiment of the present application.

[0125] like Figure 8As shown, the grid frequency reference signal network construction and audio positioning device 10 includes: an extraction module 100 , a first processing module 200 , a second processing module 200 and a positioning module 300 .

[0126] The extraction module 100 is used to extract the original audio from the multi-region online real-time multimedia data and downsample the original audio to obtain the downsampled audio.

[0127] The first processing module 200 is used to detect whether the audio has a power grid frequency, and if it is detected that the audio has a power grid frequency, process the audio to obtain a single audio power grid frequency signal.

[0128] The second processing module 300 is used to process the grid frequency signals of multiple audio frequencies in the same area to obtain a single-area grid frequency reference signal, and to process the single-area grid frequency reference signals of multiple areas to obtain a grid frequency reference signal network.

[0129] The positioning module 400 is used to process the audio to be positioned, obtain the grid frequency signal of the audio to be positioned, and calculate the correlation coefficient between the grid frequency signal to be positioned and each grid point of the grid frequency reference signal network to obtain the final positioning result of the audio according to the correlation coefficient.

[0130] Optionally, in one embodiment of the present application, the first processing module 200 includes: a first acquiring unit and a determining unit.

[0131] Wherein, the first acquisition unit is used to acquire the signal length of the audio power grid frequency signal.

[0132] The determination unit is used to calculate the noise segment rate based on the signal length, so as to determine whether the audio has a power grid frequency based on the noise segment rate.

[0133] Optionally, in one embodiment of the present application, the first processing module 200 includes: an enhancement unit, an extraction unit and a first processing unit.

[0134] The enhancement unit is used to enhance the power grid frequency signal of the audio to obtain an enhanced audio signal.

[0135] The extraction unit is used to extract the multi-harmonic power grid frequency signal of the enhanced audio signal.

[0136] The first processing unit is used to select and fuse the multi-harmonic power grid frequency signals to obtain a single audio power grid frequency signal.

[0137] Optionally, in one embodiment of the present application, the second processing module 300 includes: a construction unit and a second processing unit.

[0138] Wherein, the construction unit is used to construct a target spatial domain selection matrix based on the signal length of the power grid frequency signal.

[0139] The second processing unit is used to convert the target airspace selection matrix into a target airspace selection undirected graph, and fuse the airspace signals of the target airspace selection undirected graph to obtain a single regional power grid frequency reference signal.

[0140] Optionally, in one embodiment of the present application, the second processing module 300 includes: a second acquisition unit, a collection unit and a calculation unit.

[0141] The second acquisition unit is used to divide the total area to be located in the multiple regions into multiple grids, and acquire the grid point coordinates of the multiple grids.

[0142] The acquisition unit is used to acquire the center point coordinates of the frequency reference signal of the power grid of a single region in multiple regions.

[0143] The calculation unit is used to calculate the distance between the grid point coordinates of multiple grids and the center point coordinates, so as to obtain the grid frequency reference signal network according to the distance.

[0144] It should be noted that the aforementioned explanation of the embodiment of the grid frequency reference signal network construction and audio positioning method is also applicable to the grid frequency reference signal network construction and audio positioning device of this embodiment, and will not be repeated here.

[0145] According to the power grid frequency reference signal network construction and audio positioning device proposed in the embodiment of the present application, online real-time multimedia data can be used to construct a power grid frequency reference signal network, and the reference network can be used to locate the audio data, thereby realizing the use of online real-time multimedia data as the source of the power grid frequency reference signal, avoiding the construction cost and maintenance cost of the power grid frequency reference network, while utilizing the characteristics of a large number of online multimedia data that can be collected at multiple points, effectively ensuring the geographical resolution of power grid frequency signal collection, and improving the accuracy of the power grid frequency reference signal network. Furthermore, by using the power grid frequency reference network to locate the audio, the efficiency of audio positioning in the present application is effectively improved, the scope of positioning application is expanded, and it is convenient for actual implementation. Thus, the problem that the traditional audio positioning method in the related art needs to deploy a large number of frequency interference recorders, which is costly and complex to maintain, and also faces the problems of harsh use conditions, poor positioning accuracy, low positioning accuracy, insufficient application scope, and inability to meet the needs of audio positioning in actual applications is solved.

[0146] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device may include:

[0147] A memory 901 , a processor 902 , and a computer program stored in the memory 901 and executable on the processor 902 .

[0148] When the processor 902 executes the program, the grid frequency reference signal network construction and audio positioning method provided in the above embodiment is implemented.

[0149] Furthermore, the electronic device further comprises:

[0150] The communication interface 903 is used for communication between the memory 901 and the processor 902 .

[0151] The memory 901 is used to store computer programs that can be executed on the processor 902 .

[0152] The memory 901 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0153] If the memory 901, the processor 902 and the communication interface 903 are implemented independently, the communication interface 903, the memory 901 and the processor 902 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0154] Optionally, in a specific implementation, if the memory 901, the processor 902 and the communication interface 903 are integrated on a chip, the memory 901, the processor 902 and the communication interface 903 can communicate with each other through an internal interface.

[0155] The processor 902 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0156] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned grid frequency reference signal network construction and audio positioning method.

[0157] An embodiment of the present application also provides a computer program product, including a computer program, which can run computer instructions. When the computer instructions are executed by a processor, the power grid frequency reference signal network construction and audio positioning method provided in the embodiment of the present application is implemented.

[0158] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0159] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0160] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0161] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0162] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. If implemented by hardware, as in another embodiment, it can be implemented by any one or a combination of multiple of the following technologies known in the art: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0163] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0164] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0165] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A method for constructing a power grid frequency reference signal network and audio positioning, characterized in that: The following steps are involved: Extracting original audio from multi-region online real-time multimedia data, and downsampling the original audio to obtain downsampled audio; Detecting whether the audio has a power grid frequency, and if it is detected that the audio has the power grid frequency, processing the audio to obtain a power grid frequency signal of a single audio; Processing the grid frequency signals of multiple audio frequencies in the same region to obtain a single-region grid frequency reference signal, and processing the single-region grid frequency reference signals in multiple regions to obtain a grid frequency reference signal network; The audio to be located is processed to obtain a grid frequency signal of the audio to be located, and a correlation coefficient between the grid frequency signal to be located and each grid point of the grid frequency reference signal network is calculated to obtain a final positioning result of the audio according to the correlation coefficient.

2. The method according to claim 1, characterized in that The detecting whether the audio has a power grid frequency comprises: Obtaining the signal length of the power grid frequency signal of the audio; A noise segment rate is calculated based on the signal length to determine whether the audio has a power grid frequency based on the noise segment rate.

3. The method according to claim 1, characterized in that The processing of the audio to obtain a single audio power grid frequency signal includes: Amplifying the power grid frequency signal of the audio to obtain an enhanced audio signal; Extracting a multi-harmonic power grid frequency signal of the enhanced audio signal; The multi-harmonic power grid frequency signal is selected and fused to obtain the single audio power grid frequency signal.

4. The method according to claim 1, characterized in that: The processing of the power grid frequency signals of multiple audio frequencies in the same region to obtain a single region power grid frequency reference signal includes: Based on the signal length of the power grid frequency signal, construct a target airspace selection matrix; The target airspace selection matrix is ​​converted into a target airspace selection undirected graph, and the airspace signals of the target airspace selection undirected graph are fused to obtain the single-region power grid frequency reference signal.

5. The method according to claim 1, characterized in that: The processing of the single-region power grid frequency reference signal of multiple regions to obtain a power grid frequency reference signal network includes: Dividing the total area to be located in the multiple regions into a plurality of grids, and obtaining the coordinates of the points of the plurality of grids; Collecting the center point coordinates of the power grid frequency reference signal of the single region in the multiple regions; The distances between the grid point coordinates of the plurality of grids and the center point coordinates are calculated to obtain the grid frequency reference signal network according to the distances.

6. A power grid frequency reference signal network construction and audio positioning device, characterized in that: include: An extraction module, used for extracting original audio from multi-region online real-time multimedia data, and performing downsampling processing on the original audio to obtain downsampled audio; A first processing module is used to detect whether the audio has a power grid frequency, and if it is detected that the audio has the power grid frequency, process the audio to obtain a power grid frequency signal of a single audio; A second processing module is used to process the power grid frequency signals of multiple audio frequencies in the same region to obtain a single-region power grid frequency reference signal, and to process the single-region power grid frequency reference signals of multiple regions to obtain a power grid frequency reference signal network; The positioning module is used to process the audio to be located, obtain the power grid frequency signal of the audio to be located, and calculate the correlation coefficient between the power grid frequency signal to be located and each grid point of the power grid frequency reference signal network, so as to obtain the final positioning result of the audio according to the correlation coefficient.

7. The device according to claim 6, characterized in that The first processing module comprises: An acquisition unit, used for acquiring a signal length of a power grid frequency signal of the audio; A determination unit is configured to calculate a noise segment rate based on the signal length, so as to determine whether the audio has a power grid frequency based on the noise segment rate.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the grid frequency reference signal network construction and audio positioning method as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the grid frequency reference signal network construction and audio positioning method as described in any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed, it is used to implement the grid frequency reference signal network construction and audio positioning method as described in any one of claims 1-5.

Images