A positioning method, device and equipment of a physical speaker and a virtual speaker and a storage medium

Abstract

The application discloses a physical loudspeaker and virtual loudspeaker positioning method and device, equipment and storage medium, relates to the technical field of spatial audio, and comprises the following steps: collecting a reference signal played by a physical loudspeaker by using a microphone array, obtaining a first cross-correlation sequence through phase transformation weighting, determining a main area according to a main peak and detecting a secondary peak, and obtaining a direct sound area and a reflected sound area respectively. The signals in the two areas are respectively subjected to secondary weighting cross-correlation to obtain a relative time delay estimation value, and then the time delay difference slope is linearly fitted with the channel serial number difference to determine a first direction angle of the physical loudspeaker and a second direction angle of a virtual loudspeaker, and the distance of the physical loudspeaker and the total path length of reflection are calculated. Finally, the polar coordinate positions of the two are determined based on the above parameters, so that the positioning efficiency of the physical loudspeaker and the virtual loudspeaker is improved.

Description

Technical Field

[0001] This invention relates to the field of spatial audio technology, and in particular to a method, apparatus, device, and storage medium for locating physical and virtual loudspeakers. Background Technology

[0002] Currently, the basic working principle of reflective sound channels is as follows: the speaker first emits sound towards the side or rear wall, and then uses the reflected sound to form a virtual sound source location in the listening area. This type of solution can improve the sense of space without significantly increasing hardware costs, but positioning deviations often occur in actual deployments. The main reason is that the position, material, and placement of walls vary significantly in different home environments, and direct sound, primary reflection, and multipath reflection often coexist.

[0003] Existing technologies either use a fixed template angle or estimate time delay by taking the maximum peak in cross-correlation. The former is not robust enough to room variations, while the latter is prone to misjudgment under multipath conditions. Especially in the reflection channel, the maximum peak does not always correspond to the target reflection path; sometimes it falls on the direct component or sidelobe, thus causing virtual orientation drift.

[0004] As can be seen from the above, improving the efficiency of locating physical and virtual speakers is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method, apparatus, device, and storage medium for locating physical and virtual speakers, which can improve the efficiency of locating physical and virtual speakers during the positioning process. The specific solution is as follows:

[0006] Firstly, this application provides a method for locating physical and virtual loudspeakers, including:

[0007] Acquire the reference signal played by the physical speaker and use a microphone array to collect the recording signals of each channel;

[0008] The reference signal and the recording signals of each channel are phase-transformed and weighted to obtain a first cross-correlation sequence. The main region is determined based on the position of the main peak on each first cross-correlation sequence, and the secondary peak is detected in the main region. The direct sound region and the reflected sound region are determined based on the time delay index corresponding to the position of the secondary peak and the position of the main peak, respectively.

[0009] A second-order weighted cross-correlation is performed on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain a relative time delay estimate. Then, a linear fit is performed between each relative time delay estimate and the channel number difference to obtain the time delay difference slope.

[0010] The first direction angle of the physical speaker and the second direction angle of the virtual speaker are determined based on the respective time delay difference slopes, and the distance of the physical speaker and the total reflection path length are determined based on the first direction angle and the second direction angle; the virtual speaker is the virtual image generated by the physical speaker through reflection.

[0011] The polar coordinate positions of the physical speaker and the virtual speaker are determined based on the physical speaker distance, the first direction angle, the total reflection path length, and the second direction angle.

[0012] Optionally, acquiring the reference signal played by the physical speaker and using a microphone array to collect recording signals from each channel includes:

[0013] Acquire a reference signal played by the physical speaker; the reference signal is a discrete-time reference signal;

[0014] A linear microphone array is used to acquire recording signals for each channel corresponding to the reference signal based on a preset sampling frequency; wherein, the linear microphone array includes a preset number of microphone channels; the element spacing between each adjacent microphone channel is a preset distance;

[0015] The reference signal and the recording signal of each channel are deconvolved to obtain the spatial impulse response corresponding to each microphone channel, and an array manifold vector is constructed based on each spatial impulse response; the array manifold vector is used to characterize the directional features of the physical loudspeaker relative to the linear microphone array.

[0016] Optionally, the step of performing phase transformation weighting on the reference signal and the recording signals of each channel to obtain a first cross-correlation sequence, determining the main region based on the main peak position of each first cross-correlation sequence, and detecting secondary peaks in the main region includes:

[0017] The reference signal and the recording signals of each channel are subjected to frequency domain transformation to obtain a first frequency domain representation of the reference signal and a second frequency domain representation of each recording signal of each channel. The second frequency domain representations are then conjugated to obtain the conjugate result.

[0018] The frequency domain result is determined by using a preset phase transformation weighting operator and based on the first frequency domain representation, each of the conjugate results, a preset constant, and a preset weighting exponent. Then, the frequency domain result is subjected to an inverse frequency domain transformation to obtain the first cross-correlation sequence corresponding to each channel.

[0019] Global maximum peak detection is performed on each of the first cross-correlation sequences to obtain the corresponding main peak position and main peak amplitude. A main peak search window is established based on each main peak position as the center, so as to construct an amplitude sequence based on the main peak amplitude within each main peak search window.

[0020] The sliding energy sequence is determined based on the preset sliding window length and each amplitude sequence, and the amplitude fall-off threshold of each channel is determined based on each main peak amplitude and the preset amplitude fall-off coefficient. Then, the energy fall-off threshold is determined based on the energy at each main peak position and the preset energy fall-off coefficient.

[0021] The boundary is expanded point by point from the main peak position to the left and right sides, and the initial main region corresponding to each channel is constructed based on the current boundary expansion result, the amplitude fall-off threshold and the energy fall-off threshold.

[0022] Determine whether the initial main region meets the preset constraint conditions. If it does, set the initial main region as the target main region. If it does not, symmetrically expand the initial main region based on the preset width threshold to obtain the target main region.

[0023] Based on the target main region, a main region retention sequence and a main region suppression sequence are constructed, and secondary peaks are detected in the target main region of each channel based on each main region retention sequence and the main region suppression sequence to obtain the position of the secondary peak.

[0024] Optionally, determining the direct sound region and the reflected sound region based on the time delay indexes corresponding to the secondary peak position and the primary peak position respectively includes:

[0025] Determine the first time delay index corresponding to each of the main peak positions and the second time delay index corresponding to each of the secondary peak positions, and determine whether the second time delay index is less than the first time delay index. If the second time delay index is less than the first time delay index, then set the corresponding region as the initial direct sound region.

[0026] If the second delay index is not less than the first delay index, the corresponding region is set as the initial reflected sound region, and a protective band of a preset width is set between the initial direct sound region and the initial reflected sound region, so as to use the protective band to process the region where the initial direct sound region and the initial reflected sound region overlap, thereby obtaining the target direct sound region and the target reflected sound region.

[0027] Optionally, the step of performing a second-order weighted cross-correlation on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain a relative time delay estimate, and then performing a linear fit between each relative time delay estimate and the channel number difference to obtain the time delay difference slope, includes:

[0028] Determine the direct input signal corresponding to the direct sound region, and determine the reflected input signal corresponding to the reflected sound region;

[0029] Perform a second-weighted cross-correlation on each pair of the direct input signals to obtain a direct second-order cross-correlation sequence, detect the peak value in the direct second-order cross-correlation sequence, and then convert the first time delay index corresponding to the peak value into a first relative time delay estimate.

[0030] A linear model is established based on the first channel number difference and the first relative time delay estimate. A linear fit is performed based on the linear model to obtain the first time delay difference slope of each adjacent array element in the direct sound region.

[0031] Perform a second-weighted cross-correlation on each pair of the reflected input signals to obtain a reflection second-cross-correlation sequence, detect the peak value in the reflection second-cross-correlation sequence, and then convert the second time delay index corresponding to the peak value into a second relative time delay estimate.

[0032] A linear model is established based on the second channel number difference and the second relative time delay estimate. A linear fit is then performed based on the linear model to obtain the second time delay difference slope of each adjacent array element in the reflected sound region.

[0033] Optionally, determining the first directional angle of the physical speaker and the second directional angle of the virtual speaker based on the respective time delay difference slopes, and determining the physical speaker distance and the total reflection path length based on the first directional angle and the second directional angle, includes:

[0034] The first direction angle estimate of the physical loudspeaker is determined based on the first time delay difference slope, and the first target direction angle is determined based on the first direction angle estimate, the sound speed and the preset array element spacing.

[0035] Based on the first time delay difference slope, the direct input signal of each channel is delayed and added to enhance it, so as to obtain the direct sound enhancement response, and the physical speaker distance is determined based on the direct sound enhancement response, sound speed and preset array element spacing.

[0036] The second direction angle estimate of the physical loudspeaker is determined based on the second time delay difference slope, and the second target direction angle is determined based on the second direction angle estimate, the sound speed and the preset array element spacing;

[0037] Based on the second time delay difference slope, the reflected input signals of each channel are delayed and added together to enhance the reflected sound response. The weighted centering method is used to estimate the absolute propagation delay of the reflection on the enhanced sound response, so as to determine the total path length of the reflection based on the absolute propagation delay of the reflection and the sound speed.

[0038] Optionally, determining the polar coordinate positions of the physical speaker and the virtual speaker based on the physical speaker distance, the first directional angle, the total reflection path length, and the second directional angle includes:

[0039] Determine the difference between the total reflected path length and the physical loudspeaker distance, and determine the additional propagation path of the reflected path relative to the direct path based on the difference;

[0040] Determine the directional angle difference between the second directional angle and the first directional angle, and determine the polar radius corresponding to the virtual speaker based on the physical speaker distance, the total reflection path length, the additional propagation path, and the directional angle difference;

[0041] Using the array reference point as the origin of polar coordinates and the first direction angle as the first polar angle, and then setting the distance of the physical speaker as the first polar radius, the first polar coordinate position corresponding to the physical speaker is determined based on the origin of polar coordinates, the first polar angle and the first polar radius;

[0042] The second direction angle is set as the second polar angle, and the total reflection path length is set as the second polar radius, so as to determine the second polar coordinate position corresponding to the virtual speaker based on the second polar angle and the second polar radius.

[0043] Secondly, this application provides a positioning device for physical and virtual loudspeakers, comprising:

[0044] The signal determination module is used to acquire the reference signal played by the physical speaker and to collect the recording signals of each channel using the microphone array;

[0045] The cross-correlation sequence determination module is used to perform phase transformation weighting on the reference signal and the recording signals of each channel to obtain a first cross-correlation sequence, and to determine the main region based on the main peak position on each first cross-correlation sequence, and to detect the secondary peak in the main region, so as to determine the direct sound region and the reflected sound region based on the time delay index corresponding to the secondary peak position and the main peak position respectively.

[0046] The delay difference slope generation module is used to perform a second-weighted cross-correlation on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain a relative delay estimate, and to perform a linear fit between each relative delay estimate and the channel number difference to obtain the delay difference slope.

[0047] The orientation angle determination module is used to determine the first orientation angle of the physical speaker and the second orientation angle of the virtual speaker based on the respective time delay difference slopes, and to determine the distance of the physical speaker and the total reflection path length based on the first orientation angle and the second orientation angle; the virtual speaker is a virtual image generated by the physical speaker through reflection.

[0048] The polar coordinate position generation module is used to determine the polar coordinate positions of the physical speaker and the virtual speaker based on the physical speaker distance, the first direction angle, the total reflection path length, and the second direction angle.

[0049] Thirdly, this application provides an electronic device, comprising:

[0050] Memory, used to store computer programs;

[0051] A processor is used to execute the computer program to implement the aforementioned method for locating physical and virtual speakers.

[0052] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned method for locating physical and virtual speakers.

[0053] As can be seen from the above, before locating the physical speaker and the virtual speaker, this application needs to obtain the reference signal played by the physical speaker and collect the recording signals of each channel using a microphone array; perform phase transformation and weighting on the reference signal and the recording signals of each channel to obtain a first cross-correlation sequence, and determine the main region based on the position of the main peak on each first cross-correlation sequence, and detect the secondary peak in the main region, and determine the direct sound region and the reflected sound region based on the time delay index corresponding to the position of the secondary peak and the position of the main peak, respectively; perform second weighted cross-correlation on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain the relative time delay estimate, and perform linear fitting on each relative time delay estimate and the channel number difference to obtain the time delay difference slope; determine the first direction angle of the physical speaker and the second direction angle of the virtual speaker based on each time delay difference slope, and determine the distance of the physical speaker and the total reflection path length based on the first direction angle and the second direction angle; the virtual speaker is the virtual image generated by the physical speaker through reflection; determine the polar coordinate position of the physical speaker and the virtual speaker based on the distance of the physical speaker, the first direction angle, the total reflection path length and the second direction angle.

[0054] Therefore, this application first needs to obtain the reference signal played by the physical speaker and collect the recording signals of each channel using a microphone array; secondly, the reference signal and the recording signals of each channel are phase-transformed and weighted to obtain a first cross-correlation sequence, and the main region is determined based on the position of the main peak on each first cross-correlation sequence, and the secondary peak is detected in the main region, so as to determine the direct sound region and the reflected sound region based on the time delay index corresponding to the position of the secondary peak and the position of the main peak, respectively; then, the two direct input signals in the direct sound region and the two reflected input signals in the reflected sound region are cross-correlated twice to obtain the relative time delay estimate, and the relative time delay estimate is linearly fitted with the channel number difference to obtain the time delay difference slope; furthermore, the first direction angle of the physical speaker and the second direction angle of the virtual speaker are determined based on the time delay difference slope, so as to determine the distance of the physical speaker and the total reflection path length based on the first direction angle and the second direction angle; finally, the polar coordinate positions of the physical speaker and the virtual speaker are determined based on the distance of the physical speaker, the first direction angle, the total reflection path length and the second direction angle. This improves the efficiency of locating physical and virtual speakers, thereby enhancing the user experience. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0056] Figure 1 This is a flowchart of a method for locating physical and virtual loudspeakers disclosed in this application;

[0057] Figure 2 This is a flowchart illustrating a specific method for locating physical and virtual loudspeakers disclosed in this application;

[0058] Figure 3 This application discloses a specific flowchart of a cross-correlation and region segmentation process;

[0059] Figure 4 This is a schematic diagram of a specific process for performing secondary cross-correlation and time delay estimation disclosed in this application;

[0060] Figure 5 This is a schematic diagram of the positioning device for a physical loudspeaker and a virtual loudspeaker disclosed in this application;

[0061] Figure 6 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Currently, the basic working principle of reflective sound channels is as follows: the speaker first emits sound towards the side or rear wall, and then uses the reflected sound to form a virtual sound source location in the listening area. This type of solution often experiences positioning errors in actual deployment, mainly because the wall position, material, and placement vary significantly in different home environments, and direct sound, primary reflection, and multipath reflection often coexist. Therefore, this application provides a method for locating physical and virtual speakers, which can improve the efficiency of locating physical and virtual speakers during the positioning process.

[0064] See Figure 1 As shown, this embodiment of the invention discloses a method for locating physical speakers and virtual speakers, including:

[0065] Step S11: Obtain the reference signal played by the physical speaker and use the microphone array to collect the recording signals of each channel.

[0066] In this embodiment, the flowchart corresponding to the embodiment of this application is as follows: Figure 2 As shown: First, sound source excitation and array acquisition are performed: the reference excitation signal is played using a sound source. The excitation signal can be a frequency sweep, broadband noise, or a broadband musical segment. Subsequently, a linear microphone array is used for synchronous acquisition. Channel recording signal ,in, The spacing between adjacent array elements is Sampling rate .in, Represents the discrete-time reference signal. Indicates the first Discrete sampled signals from each microphone channel Indicates the total number of channels. Indicates the spacing between adjacent microphones. This indicates the sampling rate.

[0067] Specifically, acquiring a reference signal played by a physical speaker and collecting recording signals from each channel using a microphone array can include: acquiring a reference signal played by a physical speaker; the reference signal is a discrete-time reference signal; collecting recording signals from each channel corresponding to the reference signal using a linear microphone array based on a preset sampling frequency; wherein the linear microphone array includes a preset number of microphone channels; the element spacing between adjacent microphone channels is a preset distance; performing deconvolution processing on the reference signal and the recording signals from each channel to obtain the spatial impulse response corresponding to each microphone channel, and constructing an array manifold vector based on each spatial impulse response; the array manifold vector is used to characterize the directional features of the physical speaker relative to the linear microphone array.

[0068] Step S12: Perform phase transformation weighting on the reference signal and the recording signals of each channel to obtain a first cross-correlation sequence, determine the main region based on the main peak position on each first cross-correlation sequence, and detect the secondary peak in the main region to determine the direct sound region and the reflected sound region based on the time delay index corresponding to the secondary peak position and the main peak position, respectively.

[0069] In this embodiment, the flowchart for performing one cross-correlation and region segmentation is as follows: Figure 3 As shown: First, define a function with two inputs. Weighted cross-correlation operator:

[0070] ;

[0071] in, and They are respectively and The frequency domain representation, Indicates conjugate. For discrete time delay index, For frequency index, To prevent small constants from being divided by zero, The PHAT weighted index can take values ​​ranging from 1 to 10. .

[0072] Therefore, regarding the first The channels are:

[0073] ;

[0074] Subsequently, main peak detection is performed: in each channel The global maximum peak is detected on the weighted cross-correlation sequence to obtain the main peak index. With the amplitude of the main peak Then, energy fallback boundary detection is performed: with Establish a main peak search window around the center, and record half of the window as... And construct an amplitude sequence within that window. Then calculate the sliding energy sequence:

[0075] ;

[0076] in, The length of the energy sliding window.

[0077] Furthermore, in this embodiment of the application, the amplitude and energy at the main peak are used as reference values ​​to define the amplitude drop threshold and energy drop threshold respectively:

[0078] ;

[0079] in, This is the amplitude reduction coefficient. This is the energy fallback coefficient.

[0080] Then from Extend the boundary point by point to the left and right if and only if at some point simultaneously:

[0081] ;

[0082] And continuously satisfy no less than When sampling a point, record that point as a candidate for the main region boundary. Among these, the point on the left that satisfies the condition is recorded as... The point on the right that satisfies this condition is denoted as... Based on this, the main region is obtained. .

[0083] In this embodiment, to avoid boundary jumps caused by noise, a boundary backoff strategy can be set: if no continuous interval meeting the conditions is found on either side within the search window, a conservative boundary is adopted. or If the boundary width is less than the minimum width constraint Then according to Symmetric extension.

[0084] Subsequently, region suppression is constructed: two auxiliary sequences are constructed based on the main region: the main region preservation sequence. (Preservation within the main region, suppression outside the main region) and main region suppression sequences (Suppression within the main region, retention outside the main region). The suppression operation can use zero-value or low-gain suppression.

[0085] Specifically, phase transformation and weighting are performed on the reference signal and the recording signals of each channel to obtain a first cross-correlation sequence. The main region is determined based on the main peak position of each first cross-correlation sequence, and secondary peaks are detected within the main region. This can include: performing frequency domain transformation on the reference signal and the recording signals of each channel to obtain a first frequency domain representation of the reference signal and a second frequency domain representation of each channel's recording signal; taking the conjugate of each second frequency domain representation to obtain the conjugate result; using a preset phase transformation weighting operator and based on the first frequency domain representation, each conjugate result, a preset constant, and a preset weighting exponent to determine the frequency domain result; then performing an inverse frequency domain transformation on the frequency domain result to obtain the first cross-correlation sequence corresponding to each channel; performing global maximum peak detection on each first cross-correlation sequence to obtain the corresponding main peak position and main peak amplitude; and establishing a main peak search window centered on each main peak position to detect secondary peaks within each main peak search window. An amplitude sequence is constructed based on the amplitude of the main peak. A sliding energy sequence is determined based on a preset sliding window length and each amplitude sequence. An amplitude fall-off threshold for each channel is determined based on the amplitude of each main peak and a preset amplitude fall-off coefficient. An energy fall-off threshold is then determined based on the energy at each main peak position and a preset energy fall-off coefficient. The boundary is expanded point-by-point from the main peak position to the left and right sides. An initial main region corresponding to each channel is constructed based on the current boundary expansion result, the amplitude fall-off threshold, and the energy fall-off threshold. It is determined whether the initial main region meets preset constraints. If it does, the initial main region is set as the target main region. If not, the initial main region is symmetrically expanded based on a preset width threshold to obtain the target main region. A main region retention sequence and a main region suppression sequence are constructed based on the target main region. Secondary peaks are detected in the target main region of each channel based on each main region retention sequence and main region suppression sequence to obtain the secondary peak positions.

[0086] In this embodiment, the present application requires secondary peak detection and region determination: in Perform a peak search to obtain the secondary peak index. The region is determined by chronological order: the earlier region is defined as the direct sound region. The later region is defined as the reflected sound region. And set up protective belts. Prevent overlapping areas.

[0087] Specifically, determining the direct sound region and the reflected sound region based on the time delay indices corresponding to the secondary peak position and the primary peak position can include: determining the first time delay index corresponding to each primary peak position and the second time delay index corresponding to the secondary peak position, and determining whether the second time delay index is less than the first time delay index. If the second time delay index is less than the first time delay index, the corresponding region is set as the initial direct sound region; if the second time delay index is not less than the first time delay index, the corresponding region is set as the initial reflected sound region, and a guard band of a preset width is set between the initial direct sound region and the initial reflected sound region to process the overlapping regions of the initial direct sound region and the initial reflected sound region, thereby obtaining the target direct sound region and the target reflected sound region.

[0088] Step S13: Perform a second-order weighted cross-correlation on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain a relative time delay estimate. Then, perform a linear fit between each relative time delay estimate and the channel number difference to obtain the time delay difference slope.

[0089] In this embodiment, the flowchart for performing secondary cross-correlation and time delay estimation is shown below. Figure 4 As shown: In this embodiment of the application, each channel needs to be determined based on the region determination result, starting from the original... Weighted cross-correlation sequences are used to construct two types of quadratic cross-correlation inputs: in the direct sound region Internally retained samples form direct input In the area of ​​reflected sound Internally retained samples form reflected input Samples outside the region are suppressed.

[0090] Subsequently, this embodiment of the application requires direct signal processing: first, input reading is performed: the direct input signal output in step S2-6 is read. Then, a second cross-correlation is performed on the channel pairs. of Weighted inputs perform a second cross-correlation:

[0091] ;

[0092] Thus, the direct sound region condition is obtained. The second-order cross-correlation curve. Among them, and All by The weighted cross-correlation results are obtained in segments. This is the time delay index for quadratic cross-correlation.

[0093] Then, in this embodiment of the application, peak extraction is required: peak detection is performed on the quadratic cross-correlation curves of each channel pair, the index corresponding to the peak is read and converted into a relative delay estimate. In an alternative implementation, a quadratic function can be fitted to the peak neighborhood to obtain a subsampling-level time delay estimate.

[0094] Furthermore, the embodiments of this application require relative time delay fitting: based on the channel number difference For independent variables, estimate the relative time delay (time delay difference). Establish a linear model with the dependent variable as the dependent variable:

[0095] ;

[0096] It is worth mentioning that, in this embodiment, robust fitting (including but not limited to weighted least squares and Huber robust fitting) is used to solve all channel pairs of samples to obtain an estimate of the relative time delay slope of the direct sound region. .

[0097] Subsequently, the embodiments of this application need to be based on Calculate the virtual direction:

[0098] ;

[0099] in, This represents the slope of the time delay difference between adjacent array elements obtained by fitting the direct sound region (the unit can be s / array element). This represents the physical loudspeaker's directional angle estimate. It indicates the speed of sound.

[0100] Then, based on the physical direction angle, a direct sound region compensation delay is constructed for each channel. And perform delay summation to obtain the enhanced response in the direct sound region. And in The initial peak detection will be conducted, and the earliest peak meeting the conditions will be located within the main peak's forward window. The peak is considered the direct peak. Desirable If no condition is met, then the main peak is taken.

[0101] Finally, the embodiments of this application require a direct-access peak index. Determine the absolute delay of direct access .in, This is the latency index corresponding to the direct peak. This is to achieve absolute propagation delay.

[0102] In this embodiment, the present application requires the construction of a multi-constraint optimization model: reading the input signal; reading the reflected input signal of the output. Similarly, in this embodiment, the direct input needs to be replaced with a reflected input to obtain the result under the conditions of the reflected region. Secondary cross-correlation curve.

[0103] Then, peak detection is performed on the quadratic cross-correlation curve of the reflection region, the peak index is read and converted to obtain the relative time delay estimate. In an optional implementation, embodiments of this application can perform quadratic function fitting on the peak neighborhood to obtain sub-sampling level time delay estimation. Similarly, embodiments of this application can perform relative time delay estimation sample solving on the reflection branch to obtain the relative time delay slope of the reflection region. .

[0104] Specifically, a weighted cross-correlation is performed on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain relative time delay estimates. A linear fit is then performed between each relative time delay estimate and the channel number difference to obtain the time delay difference slope. This process may include: determining the direct input signal corresponding to the direct sound region and the reflected input signal corresponding to the reflected sound region; performing a weighted cross-correlation on every two direct input signals to obtain a direct secondary cross-correlation sequence; detecting peak values ​​in the direct secondary cross-correlation sequence; and then converting the first time delay index corresponding to the peak value into a first relative time delay estimate. The calculation process involves: establishing a linear model based on the first channel number difference and the first relative time delay estimate; performing linear fitting based on the linear model to obtain the slope of the first time delay difference between adjacent array elements in the direct sound region; performing a second-order weighted cross-correlation on each pair of reflected input signals to obtain a reflection second-order cross-correlation sequence; detecting peak values ​​in the reflection second-order cross-correlation sequence; and then converting the second time delay index corresponding to the peak value into a second relative time delay estimate; establishing a linear model based on the second channel number difference and the second relative time delay estimate; and performing linear fitting based on the linear model to obtain the slope of the second time delay difference between adjacent array elements in the reflected sound region.

[0105] Step S14: Determine the first direction angle of the physical speaker and the second direction angle of the virtual speaker based on the respective time delay difference slopes, and determine the distance of the physical speaker and the total reflection path length based on the first direction angle and the second direction angle; the virtual speaker is the virtual image generated by the physical speaker through reflection.

[0106] In this embodiment, the direction calculation method needs to be performed, that is, the reflection branch slope is substituted. To obtain the virtual sound source direction angle estimate Subsequently, based on the time delay difference between adjacent array elements obtained from the fitting... Construct time compensation for the reflection regions of each channel and perform delay summation to form an enhanced response in the reflection regions, and then enhance the response in the reflection regions. Upper determination of reflection delay protection window The reflection delay index is estimated using the in-window weighted centering method. Among them, the center-weighted index The possible values ​​are .Depend on The absolute time delay of the reflection is calculated. , The absolute propagation delay is used for reflection. In an optional implementation, the in-window peak index can also be used for delay conversion.

[0107] Furthermore, this embodiment requires propagation distance calculation: a virtual loudspeaker geometric model is used, and the physical loudspeaker position is set as follows: The virtual speaker position is The array reference point is The propagation distance is calculated based on the absolute time delay:

[0108] ;

[0109] in, and These represent the direct and reflected propagation distances, respectively. It indicates the speed of sound.

[0110] Subsequently, the virtual speaker distance is calculated: with the array reference point as the origin of the polar coordinates, the physical speaker radius is calculated as follows:

[0111] ;

[0112] Additional distance is calculated as follows:

[0113] ;

[0114] in, This represents the additional propagation path of the reflected image relative to the direct path. Because... Characterizing the total propagation distance of the reflection path, it is not directly equivalent to the radial distance from the array to the virtual speaker; the virtual speaker's polar radius... Depend on , and Further calculations are performed using geometric relationships.

[0115] Specifically, determining the first directional angle of the physical loudspeaker and the second directional angle of the virtual loudspeaker based on the slopes of each time delay difference, and then determining the distance of the physical loudspeaker and the total reflection path length based on the first and second directional angles, can include: determining an estimated value of the first directional angle of the physical loudspeaker based on the slope of the first time delay difference, and determining a first target directional angle based on the estimated value of the first directional angle, the speed of sound, and a preset array element spacing; performing delay summation enhancement on the direct input signals of each channel based on the slope of the first time delay difference to obtain a direct sound enhancement response, and determining the distance of the physical loudspeaker based on the direct sound enhancement response, the speed of sound, and the preset array element spacing; determining an estimated value of the second directional angle of the physical loudspeaker based on the slope of the second time delay difference, and determining a second target directional angle based on the estimated value of the second directional angle, the speed of sound, and the preset array element spacing; performing delay summation enhancement on the reflected input signals of each channel based on the slope of the second time delay difference to obtain a reflected sound enhancement response, estimating the absolute propagation delay of the reflection on the reflected sound enhancement response using a weighted centering method, and determining the total reflection path length based on the absolute propagation delay of the reflection and the speed of sound.

[0116] Step S15: Determine the polar coordinate positions of the physical speaker and the virtual speaker based on the physical speaker distance, the first direction angle, the total reflection path length, and the second direction angle.

[0117] In this embodiment, (Note:) Then the virtual loudspeaker radius can be written as:

[0118] ;

[0119] Similarly, substitution We can obtain:

[0120] ;

[0121] It is worth mentioning that when the denominator is close to 0, a threshold protection mechanism can be used to fall back to the iterative solution or a historical smoothed value. Subsequently, the polar coordinates of the physical speaker are output. With virtual speaker polar coordinates .

[0122] In one specific implementation, the typical parameter range in the embodiments of this application is: system sampling rate. The range is 16kHz to 96kHz, and the spacing between adjacent elements in the microphone array is... Take a measurement from 3cm to 20cm, and use the PHAT weighted index. Select 0 to 1, main peak search half window The energy sliding window length is set from 0.2ms to 2ms. Amplitude drop-off coefficient, ranging from 0.2ms to 1ms. The energy fallback coefficient is set between 0.15 and 0.35. Take a value of 0.1 to 0.3, for the regional protection zone. The threshold for the first peak ratio is set between 0.2ms and 1ms. Take a value between 0.3 and 0.7.

[0123] Specifically, determining the polar coordinate positions of the physical speaker and the virtual speaker based on the physical speaker distance, the first directional angle, the total reflection path length, and the second directional angle can include: determining the difference between the total reflection path length and the physical speaker distance, and using this difference to determine the additional propagation path of the reflected path relative to the direct path; determining the directional angle difference between the second directional angle and the first directional angle, and using the physical speaker distance, the total reflection path length, the additional propagation path, and the directional angle difference to determine the polar radius corresponding to the virtual speaker; using the array reference point as the origin of the polar coordinates, and the first directional angle as the first polar angle, and then setting the physical speaker distance as the first polar radius, to determine the first polar coordinate position corresponding to the physical speaker based on the origin of the polar coordinates, the first polar angle, and the first polar radius; setting the second directional angle as the second polar angle, and setting the total reflection path length as the second polar radius, to determine the second polar coordinate position corresponding to the virtual speaker based on the second polar angle and the second polar radius.

[0124] It is worth mentioning that the system can consist of a reference excitation module and a multi-channel acquisition module. Weighted cross-correlation module, main peak and energy drop region detection module, regional suppression and secondary peak detection module, based on The system consists of a quadratic cross-correlation and direction fitting module, a delay summation and first peak detection module, and a geometric solution module. These modules can be deployed on a DSP, MCU, CPU, or a combination thereof, or implemented through program instructions in a non-transitory computer-readable storage medium.

[0125] As can be seen from the above, the embodiments of this application first need to obtain the reference signal played by the physical speaker and collect the recording signals of each channel using a microphone array; secondly, the reference signal and the recording signals of each channel are phase-transformed and weighted to obtain a first cross-correlation sequence, and the main region is determined based on the position of the main peak on each first cross-correlation sequence, and the secondary peak is detected in the main region, so as to determine the direct sound region and the reflected sound region based on the time delay index corresponding to the position of the secondary peak and the position of the main peak, respectively; then, the two direct input signals in the direct sound region and the two reflected input signals in the reflected sound region are respectively subjected to second weighted cross-correlation to obtain the relative time delay estimate, and the relative time delay estimate is linearly fitted with the channel number difference to obtain the time delay difference slope; furthermore, the first direction angle of the physical speaker and the second direction angle of the virtual speaker are determined based on the time delay difference slope, so as to determine the distance of the physical speaker and the total reflection path length based on the first direction angle and the second direction angle; finally, the polar coordinate positions of the physical speaker and the virtual speaker are determined based on the distance of the physical speaker, the first direction angle, the total reflection path length and the second direction angle. This improves the efficiency of locating physical and virtual speakers, thereby enhancing the user experience.

[0126] Accordingly, see Figure 5 As shown, this application also provides a positioning device for physical speakers and virtual speakers, including:

[0127] The signal determination module 11 is used to acquire the reference signal played by the physical speaker and to collect the recording signals of each channel using the microphone array;

[0128] The cross-correlation sequence determination module 12 is used to perform phase transformation weighting on the reference signal and the recording signal of each channel to obtain a first cross-correlation sequence, and determine the main region based on the main peak position on each first cross-correlation sequence, and detect the secondary peak in the main region, so as to determine the direct sound region and the reflected sound region based on the time delay index corresponding to the secondary peak position and the main peak position respectively.

[0129] The delay difference slope generation module 13 is used to perform a second weighted cross-correlation on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain a relative delay estimate, and to perform a linear fit between each relative delay estimate and the channel number difference to obtain the delay difference slope.

[0130] The directional angle determination module 14 is used to determine the first directional angle of the physical speaker and the second directional angle of the virtual speaker based on the respective time delay difference slopes, and to determine the distance of the physical speaker and the total reflection path length based on the first directional angle and the second directional angle; the virtual speaker is a virtual image generated by the physical speaker through reflection.

[0131] The polar coordinate position generation module 15 is used to determine the polar coordinate positions of the physical speaker and the virtual speaker based on the physical speaker distance, the first direction angle, the total reflection path length and the second direction angle.

[0132] In some specific embodiments, the signal determination module 11 may specifically include:

[0133] A reference signal determination unit is used to acquire a reference signal played by a physical loudspeaker; the reference signal is a discrete-time reference signal.

[0134] The recording signal acquisition unit is used to acquire recording signals of each channel corresponding to the reference signal using a linear microphone array and based on a preset sampling frequency; wherein, the linear microphone array includes a preset number of microphone channels; the array element spacing between each adjacent microphone channel is a preset distance;

[0135] An array manifold vector construction unit is used to perform deconvolution processing on the reference signal and the recording signal of each channel to obtain the spatial impulse response corresponding to each microphone channel, and construct an array manifold vector based on each spatial impulse response; the array manifold vector is used to characterize the directional features of the physical loudspeaker relative to the linear microphone array.

[0136] In some specific embodiments, the cross-correlation sequence determination module 12 may specifically include:

[0137] The conjugate result determination unit is used to perform frequency domain transformation processing on the reference signal and the recording signal of each channel respectively to obtain a first frequency domain representation of the reference signal and a second frequency domain representation of each channel recording signal, and to take the conjugate of each second frequency domain representation to obtain a conjugate result.

[0138] The inverse frequency domain transformation unit is used to determine the frequency domain result by using a preset phase transformation weighting operator and based on the first frequency domain representation, each of the conjugate results, a preset constant, and a preset weighting exponent, and then perform an inverse frequency domain transformation on the frequency domain result to obtain the first cross-correlation sequence corresponding to each channel;

[0139] An amplitude sequence construction unit is used to perform global maximum peak detection on each of the first cross-correlation sequences to obtain the corresponding main peak position and main peak amplitude, and to establish a main peak search window based on each main peak position to construct an amplitude sequence based on the main peak amplitude within each main peak search window.

[0140] The sliding energy sequence determination unit is used to determine the sliding energy sequence based on the preset sliding window length and each of the amplitude sequences, and to determine the amplitude fall-off threshold of each channel based on each of the main peak amplitudes and the preset amplitude fall-off coefficient, and then to determine the energy fall-off threshold based on the energy at each of the main peak positions and the preset energy fall-off coefficient.

[0141] The initial main region determination unit is used to expand the boundary point by point from the main peak position to the left and right sides, and construct the initial main region corresponding to each channel based on the current boundary expansion result, the amplitude fall-off threshold and the energy fall-off threshold;

[0142] The target main region determination unit is used to determine whether the initial main region meets the preset constraint conditions. If it does, the initial main region is set as the target main region. If it does not, the initial main region is symmetrically expanded based on the preset width threshold to obtain the target main region.

[0143] The secondary peak position determination unit is used to construct a main region retention sequence and a main region suppression sequence based on the target main region, and to detect secondary peaks in the target main region of each channel based on each main region retention sequence and the main region suppression sequence to obtain the secondary peak position.

[0144] In some specific embodiments, the cross-correlation sequence determination module 12 may specifically include:

[0145] The delay index determination unit is used to determine the first delay index corresponding to each of the main peak positions and the second delay index corresponding to each of the secondary peak positions, and to determine whether the second delay index is less than the first delay index. If the second delay index is less than the first delay index, the corresponding region is set as the initial direct sound region.

[0146] The protective band setting unit is used to set the corresponding area as the initial reflected sound area if the second time delay index is not less than the first time delay index, and to set a protective band of a preset width between the initial direct sound area and the initial reflected sound area, so as to use the protective band to process the overlapping area between the initial direct sound area and the initial reflected sound area to obtain the target direct sound area and the target reflected sound area.

[0147] In some specific embodiments, the delay difference slope generation module 13 may specifically include:

[0148] An input signal determination unit is used to determine the direct input signal corresponding to the direct sound region and to determine the reflected input signal corresponding to the reflected sound region.

[0149] The first relative time delay estimation unit is used to perform a second weighted cross-correlation on each pair of the direct input signals to obtain a direct second cross-correlation sequence, detect the peak value in the direct second cross-correlation sequence, and then convert the first time delay index corresponding to the peak value into a first relative time delay estimation value.

[0150] The first time delay difference slope determination unit is used to establish a linear model based on the first channel number difference and the first relative time delay estimate, and to perform linear fitting based on the linear model to obtain the first time delay difference slope of each adjacent array element in the direct sound region.

[0151] The second relative time delay estimation unit is used to perform a second weighted cross-correlation on each pair of the reflected input signals to obtain a reflection second cross-correlation sequence, detect the peak value in the reflection second cross-correlation sequence, and then convert the second time delay index corresponding to the peak value into a second relative time delay estimation value.

[0152] The second time delay difference slope determination unit is used to establish a linear model based on the second channel number difference and the second relative time delay estimate, and to perform linear fitting based on the linear model to obtain the second time delay difference slope of each adjacent array element in the reflected sound region.

[0153] In some specific embodiments, the orientation angle determination module 14 may specifically include:

[0154] The first target direction angle determination unit is used to determine the first direction angle estimate of the physical loudspeaker based on the first time delay difference slope, and to determine the first target direction angle based on the first direction angle estimate, the sound speed and the preset array element spacing.

[0155] The physical speaker distance determination unit is used to perform delay summation enhancement on the direct input signal of each channel based on the first time delay difference slope to obtain the direct sound enhancement response, and to determine the physical speaker distance based on the direct sound enhancement response, sound speed and preset array element spacing.

[0156] The second target direction angle determination unit is used to determine the second direction angle estimate of the physical loudspeaker based on the second time delay difference slope, and to determine the second target direction angle based on the second direction angle estimate, the sound speed and the preset array element spacing;

[0157] The total reflection path length determination unit is used to perform delay summation enhancement on the reflected input signals of each channel based on the second time delay difference slope to obtain the reflected sound enhancement response, and to estimate the absolute reflection propagation delay on the reflected sound enhancement response using the weighted centering method, so as to determine the total reflection path length based on the absolute reflection propagation delay and the sound speed.

[0158] In some specific embodiments, the polar coordinate position generation module 15 may specifically include:

[0159] An additional propagation path determination unit is used to determine the difference between the total reflection path length and the physical loudspeaker distance, so as to determine the additional propagation path of the reflection path relative to the direct path based on the difference;

[0160] The polar radius determination unit is used to determine the directional angle difference between the second directional angle and the first directional angle, and to determine the polar radius corresponding to the virtual speaker based on the physical speaker distance, the total reflection path length, the additional propagation path, and the directional angle difference.

[0161] The first polar coordinate position determination unit is used to take the array reference point as the origin of polar coordinates and the first direction angle as the first polar angle, and then set the distance of the physical speaker as the first polar radius, so as to determine the first polar coordinate position corresponding to the physical speaker based on the origin of polar coordinates, the first polar angle and the first polar radius;

[0162] The second polar coordinate position determination unit is used to set the second direction angle as the second polar angle and the total reflection path length as the second polar radius, so as to determine the second polar coordinate position corresponding to the virtual speaker based on the second polar angle and the second polar radius.

[0163] Furthermore, embodiments of this application also disclose an electronic device, Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the physical and virtual speaker positioning methods disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a computer.

[0164] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0165] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0166] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the physical speaker and virtual speaker positioning method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0167] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned method for locating physical and virtual speakers. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0168] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0169] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0170] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0171] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0172] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for locating physical and virtual loudspeakers, characterized in that, include: Acquire the reference signal played by the physical speaker and use a microphone array to collect the recording signals of each channel; The reference signal and the recording signals of each channel are phase-transformed and weighted to obtain a first cross-correlation sequence. The main region is determined based on the position of the main peak on each first cross-correlation sequence, and the secondary peak is detected in the main region. The direct sound region and the reflected sound region are determined based on the time delay index corresponding to the position of the secondary peak and the position of the main peak, respectively. Specifically, the reference signal and each of the channel recording signals are subjected to frequency domain transformation processing to obtain a first frequency domain representation of the reference signal and a second frequency domain representation of each of the channel recording signals. The second frequency domain representations are then conjugated to obtain conjugated results. A preset phase transformation weighting operator is used to determine the frequency domain result based on the first frequency domain representation, each of the conjugate results, a preset constant, and a preset weighting exponent. Then, the frequency domain result is subjected to an inverse frequency domain transformation to obtain the first cross-correlation sequence corresponding to each channel. Global maximum peak detection is performed on each of the first cross-correlation sequences to obtain the corresponding main peak position and main peak amplitude. A main peak search window is established based on each main peak position as the center to construct an amplitude sequence based on the main peak amplitude within each main peak search window. The sliding energy sequence is determined based on the preset sliding window length and each amplitude sequence, and the amplitude fall-off threshold of each channel is determined based on each main peak amplitude and the preset amplitude fall-off coefficient. Then, the energy fall-off threshold is determined based on the energy at each main peak position and the preset energy fall-off coefficient. The boundary is expanded point by point from the main peak position to the left and right sides, and an initial main region corresponding to each channel is constructed based on the current boundary expansion result, the amplitude fall-off threshold, and the energy fall-off threshold. It is determined whether the initial main region meets the preset constraint conditions. If it does, the initial main region is set as the target main region; if not, the initial main region is symmetrically expanded based on the preset width threshold to obtain the target main region. A main region retention sequence and a main region suppression sequence are constructed based on the target main region, and secondary peaks are detected in the target main region of each channel based on each main region retention sequence and the main region suppression sequence to obtain the secondary peak positions. The positions of each secondary peak are then determined. The system calculates a first time delay index corresponding to the main peak position and a second time delay index corresponding to the secondary peak position, and determines whether the second time delay index is less than the first time delay index. If the second time delay index is less than the first time delay index, the corresponding region is set as the initial direct sound region. If the second time delay index is not less than the first time delay index, the corresponding region is set as the initial reflected sound region. A protective band of a preset width is set between the initial direct sound region and the initial reflected sound region to process the overlapping regions of the initial direct sound region and the initial reflected sound region, thereby obtaining the target direct sound region and the target reflected sound region. A second-order weighted cross-correlation is performed on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain a relative time delay estimate. Then, a linear fit is performed between each relative time delay estimate and the channel number difference to obtain the time delay difference slope. The first direction angle of the physical speaker and the second direction angle of the virtual speaker are determined based on the time delay difference slopes, and the distance of the physical speaker and the total reflection path length are determined based on the first direction angle and the second direction angle. The virtual loudspeaker is a virtual image generated by the physical loudspeaker through reflection. The polar coordinate positions of the physical speaker and the virtual speaker are determined based on the physical speaker distance, the first direction angle, the total reflection path length, and the second direction angle.

2. The method for locating physical and virtual loudspeakers according to claim 1, characterized in that, The process of acquiring the reference signal played by the physical speaker and collecting the recording signals of each channel using a microphone array includes: Acquire a reference signal played by the physical speaker; the reference signal is a discrete-time reference signal; A linear microphone array is used to acquire recording signals for each channel corresponding to the reference signal based on a preset sampling frequency; wherein, the linear microphone array includes a preset number of microphone channels; the element spacing between each adjacent microphone channel is a preset distance; The reference signal and the recording signal of each channel are deconvolved to obtain the spatial impulse response corresponding to each microphone channel, and an array manifold vector is constructed based on each spatial impulse response; the array manifold vector is used to characterize the directional features of the physical loudspeaker relative to the linear microphone array.

3. The method for locating physical and virtual loudspeakers according to claim 1, characterized in that, The process involves performing a second-order weighted cross-correlation on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain a relative time delay estimate. Then, a linear fit is performed between each relative time delay estimate and the channel number difference to obtain the time delay difference slope, including: Determine the direct input signal corresponding to the direct sound region, and determine the reflected input signal corresponding to the reflected sound region; Perform a second-weighted cross-correlation on each pair of the direct input signals to obtain a direct second-order cross-correlation sequence, detect the peak value in the direct second-order cross-correlation sequence, and then convert the first time delay index corresponding to the peak value into a first relative time delay estimate. A linear model is established based on the first channel number difference and the first relative time delay estimate. A linear fit is performed based on the linear model to obtain the first time delay difference slope of each adjacent array element in the direct sound region. Perform a second-weighted cross-correlation on each pair of the reflected input signals to obtain a reflection second-cross-correlation sequence, detect the peak value in the reflection second-cross-correlation sequence, and then convert the second time delay index corresponding to the peak value into a second relative time delay estimate. A linear model is established based on the second channel number difference and the second relative time delay estimate. A linear fit is then performed based on the linear model to obtain the second time delay difference slope of each adjacent array element in the reflected sound region.

4. The method for locating physical and virtual loudspeakers according to claim 3, characterized in that, The step of determining the first directional angle of the physical speaker and the second directional angle of the virtual speaker based on the respective time delay difference slopes, and determining the physical speaker distance and the total reflection path length based on the first directional angle and the second directional angle, includes: The first direction angle estimate of the physical loudspeaker is determined based on the first time delay difference slope, and the first target direction angle is determined based on the first direction angle estimate, the sound speed and the preset array element spacing. Based on the first time delay difference slope, the direct input signal of each channel is delayed and added to enhance it, so as to obtain the direct sound enhancement response, and the physical speaker distance is determined based on the direct sound enhancement response, sound speed and preset array element spacing. The second direction angle estimate of the physical loudspeaker is determined based on the second time delay difference slope, and the second target direction angle is determined based on the second direction angle estimate, the sound speed and the preset array element spacing; Based on the second time delay difference slope, the reflected input signals of each channel are delayed and added together to enhance the reflected sound response. The weighted centering method is used to estimate the absolute propagation delay of the reflection on the enhanced sound response, so as to determine the total path length of the reflection based on the absolute propagation delay of the reflection and the sound speed.

5. The method for locating physical and virtual loudspeakers according to any one of claims 1 to 4, characterized in that, Determining the polar coordinate positions of the physical speaker and the virtual speaker based on the physical speaker distance, the first directional angle, the total reflection path length, and the second directional angle includes: Determine the difference between the total reflected path length and the physical loudspeaker distance, and determine the additional propagation path of the reflected path relative to the direct path based on the difference; Determine the directional angle difference between the second directional angle and the first directional angle, and determine the polar radius corresponding to the virtual speaker based on the physical speaker distance, the total reflection path length, the additional propagation path, and the directional angle difference; Using the array reference point as the origin of polar coordinates and the first direction angle as the first polar angle, and then setting the distance of the physical speaker as the first polar radius, the first polar coordinate position corresponding to the physical speaker is determined based on the origin of polar coordinates, the first polar angle and the first polar radius; The second direction angle is set as the second polar angle, and the total reflection path length is set as the second polar radius, so as to determine the second polar coordinate position corresponding to the virtual speaker based on the second polar angle and the second polar radius.

6. A positioning device for a physical loudspeaker and a virtual loudspeaker, characterized in that, include: The signal determination module is used to acquire the reference signal played by the physical speaker and to collect the recording signals of each channel using the microphone array; The cross-correlation sequence determination module is used to perform phase transformation weighting on the reference signal and the recording signals of each channel to obtain a first cross-correlation sequence, and to determine the main region based on the main peak position on each first cross-correlation sequence, and to detect the secondary peak in the main region, so as to determine the direct sound region and the reflected sound region based on the time delay index corresponding to the secondary peak position and the main peak position respectively. Specifically, the reference signal and each of the channel recording signals are subjected to frequency domain transformation processing to obtain a first frequency domain representation of the reference signal and a second frequency domain representation of each of the channel recording signals. The second frequency domain representations are then conjugated to obtain conjugated results. A preset phase transformation weighting operator is used to determine the frequency domain result based on the first frequency domain representation, each of the conjugate results, a preset constant, and a preset weighting exponent. Then, the frequency domain result is subjected to an inverse frequency domain transformation to obtain the first cross-correlation sequence corresponding to each channel. Global maximum peak detection is performed on each of the first cross-correlation sequences to obtain the corresponding main peak position and main peak amplitude. A main peak search window is established based on each main peak position as the center to construct an amplitude sequence based on the main peak amplitude within each main peak search window. The sliding energy sequence is determined based on the preset sliding window length and each amplitude sequence, and the amplitude fall-off threshold of each channel is determined based on each main peak amplitude and the preset amplitude fall-off coefficient. Then, the energy fall-off threshold is determined based on the energy at each main peak position and the preset energy fall-off coefficient. The boundary is expanded point by point from the main peak position to the left and right sides, and an initial main region corresponding to each channel is constructed based on the current boundary expansion result, the amplitude fall-off threshold, and the energy fall-off threshold. It is determined whether the initial main region meets the preset constraint conditions. If it does, the initial main region is set as the target main region; if not, the initial main region is symmetrically expanded based on the preset width threshold to obtain the target main region. A main region retention sequence and a main region suppression sequence are constructed based on the target main region, and secondary peaks are detected in the target main region of each channel based on each main region retention sequence and the main region suppression sequence to obtain the secondary peak positions. The positions of each secondary peak are then determined. The system calculates a first time delay index corresponding to the main peak position and a second time delay index corresponding to the secondary peak position, and determines whether the second time delay index is less than the first time delay index. If the second time delay index is less than the first time delay index, the corresponding region is set as the initial direct sound region. If the second time delay index is not less than the first time delay index, the corresponding region is set as the initial reflected sound region. A protective band of a preset width is set between the initial direct sound region and the initial reflected sound region to process the overlapping regions of the initial direct sound region and the initial reflected sound region, thereby obtaining the target direct sound region and the target reflected sound region. The delay difference slope generation module is used to perform a second-weighted cross-correlation on every two direct input signals in the direct sound region and every two reflected input signals in the reflected sound region to obtain a relative delay estimate, and to perform a linear fit between each relative delay estimate and the channel number difference to obtain the delay difference slope. The orientation angle determination module is used to determine the first orientation angle of the physical speaker and the second orientation angle of the virtual speaker based on the respective time delay difference slopes, and to determine the physical speaker distance and the total reflection path length based on the first orientation angle and the second orientation angle; The virtual loudspeaker is a virtual image generated by the physical loudspeaker through reflection. The polar coordinate position generation module is used to determine the polar coordinate positions of the physical speaker and the virtual speaker based on the physical speaker distance, the first direction angle, the total reflection path length, and the second direction angle.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method for locating physical and virtual speakers as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the method for locating physical and virtual speakers as described in any one of claims 1 to 5.

Images