A method, device, medium and product for sound source azimuth determination of a dual-microphone array

Abstract

This application provides a method, device, medium, and product for determining the location of a sound source using a dual-microphone array. The method determines whether the sound source location tracking process has entered the end-fire acquisition state based on the delay measurement value, smooth delay state value, and measurement reliability of the current audio frame. When the sound source location tracking process enters the end-fire acquisition state, the method adjusts the state estimation parameters and updates the smooth delay state value based on the adjusted state estimation parameters. Then, based on the updated smooth delay state value and the distance between the two microphones in the dual-microphone array, the location information of the sound source is determined. This ensures that the smooth delay state value responds promptly to reliable end-fire delay measurements, improving the accuracy of sound source location determination and tracking response capability of the dual-microphone array.

Description

Technical Field

[0001] This application relates to the field of audio data processing technology, and more specifically, to a method, device, medium, and product for determining the location of a sound source using a dual-microphone array. Background Technology

[0002] With the development of smart speakers, conference terminals, robots, and other voice-interactive devices, sound source localization technology is widely used in scenarios such as voice pickup control, voice enhancement, interactive object recognition, and device orientation control. Dual-microphone arrays, due to their smaller number of microphones, offer advantages such as compact structure, low computational load, and ease of deployment on microcontrollers or on-chip systems. Traditional dual-microphone sound source localization methods typically determine the sound source location based on the time difference between the audio signals collected by the two microphones.

[0003] When a sound source approaches the end-fire direction of a dual-microphone array, small time delay measurement disturbances can be amplified into large azimuth changes by geometric mapping. Therefore, traditional techniques often suppress azimuth jitter by reducing the impact of end-fire measurements on the smooth time delay state update results. However, when reliable time delay measurements indicate that the sound source is further approaching the end-fire direction or switching from one side of the array to the other, traditional methods will excessively suppress effective measurements, making it difficult for the smooth time delay state to converge to the current time delay measurement in a timely manner. This results in response lag or azimuth deviation in the sound source azimuth tracking results, reducing the accuracy of azimuth determination in the end-fire direction.

[0004] In response to this situation, an improved solution is urgently needed to enhance the accuracy and efficiency of sound source location determination in dual-microphone arrays. Summary of the Invention

[0005] The purpose of this application is to provide a solution that can at least solve one or more of the above or other problems.

[0006] In one aspect of this application, a method for determining the location of a sound source using a dual-microphone array is provided, characterized in that the dual-microphone array includes two microphones spaced apart, and the method includes: Based on the current audio frame's delay measurement value, smooth delay state value, and measurement reliability, determine whether the sound source orientation tracking process has entered the end-fire acquisition state. When the sound source orientation tracking process enters the end-fire acquisition state, the state estimation parameters are adjusted, and the smooth delay state value is updated based on the adjusted state estimation parameters. The state estimation parameters are used to control the degree of influence of the delay measurement value on the update result of the smooth delay state value. Based on the updated smooth delay state value and the microphone spacing of the dual-microphone array, the directional information of the sound source is determined.

[0007] Optionally, in some examples, when the measurement confidence level meets a preset confidence level requirement and the sound source orientation tracking process meets at least one of the end-fire enhancement condition and the translateral change condition, the sound source orientation tracking process is determined to enter the end-fire acquisition state.

[0008] Optionally, in some examples, the end-fire enhancement condition is characterized by the sound source orientation corresponding to the delay measurement value being further closer to the end-fire direction than the sound source orientation corresponding to the smooth delay state value, wherein the end-fire direction is the sound source direction extending along the spacing direction of the two microphones.

[0009] Optionally, in some examples, the cross-side variation condition represents that the sound source orientation corresponding to the time delay measurement value and the smooth time delay state value is located on different sides of the dual-microphone array.

[0010] Optionally, in some examples, before determining whether the sound source location tracking process has entered the end-fire acquisition state based on the delay measurement value, smooth delay state value, and measurement confidence level of the current audio frame, the method further includes: Based on the ratio of sound speed to the distance between the two microphones, the time delay measurement value and the smooth time delay state value are normalized to obtain the time delay measurement projection and the smooth time delay projection.

[0011] Optionally, in some examples, the end-fire enhancement conditions include: The difference between the absolute value of the time delay measurement projection and the absolute value of the smooth time delay projection reaches a preset increment value, and the absolute value of the time delay measurement projection is greater than a preset end-shot acquisition threshold.

[0012] Optionally, in some examples, the translateral variation condition includes: The sign of the time delay measurement projection is opposite to that of the smooth time delay projection, and the absolute value of the time delay measurement projection is greater than a preset cross-side change threshold.

[0013] Optionally, in some examples, adjusting the state estimation parameters includes increasing the degree of influence of the delay measurement on the smooth delay state value update; The degree of influence of increasing the time delay measurement value on the smooth time delay state value update includes one or more combinations of the following operations: Increase the state uncertainty corresponding to the smoothed delay state value; Increase the process noise corresponding to the state prediction process of the smoothed delay state value; Reduce the measurement noise corresponding to the time delay measurement value; Increase the state update gain corresponding to the time delay measurement value; Increase the measurement weight corresponding to the time delay measurement value; Reduce the noise amplification factor set for end-fire measurements.

[0014] Optionally, in some examples, after determining that the sound source orientation tracking process has entered the end-fire acquisition state, the method further includes: When the smooth delay state value meets the preset end-fire locking condition, the sound source azimuth tracking process is determined to enter the end-fire holding state; the end-fire holding state indicates that the smooth delay state value is within the preset delay range corresponding to the end-fire direction.

[0015] Optionally, in some examples, the end-fire locking condition includes: The absolute value of the smoothed delay projection is greater than a preset end-fire holding threshold; and The state uncertainty of the smooth delay state value is less than the preset locking threshold.

[0016] Optionally, in some examples, after determining that the sound source orientation tracking process has entered the end-fire hold state, the method further includes: Determine the information between the time delay measurement value and the smoothed time delay state value before the update, as well as the direction of change corresponding to the information; The innovation is constrained based on the direction of change and the upper limit of the innovation constraint; The smoothed delay state value is updated based on the constrained information.

[0017] Optionally, in some examples, the news represents the difference between the delay measurement and the smoothed delay state value before the update, and the direction of change represents the trend of change of the absolute value of the smoothed delay state value when the smoothed delay state value is updated based on the news.

[0018] Optionally, in some examples, constraining the innovation based on the direction of change and the upper limit of the innovation constraint includes: When the direction of change is the first direction of change, the innovation is constrained based on the upper limit of the first innovation constraint; the first direction of change represents the direction of change in which the orientation of the sound source changes toward the end-emitting direction, causing the absolute value of the smooth time delay state value to increase.

[0019] Optionally, in some examples, constraining the innovation based on the direction of change and the upper limit of the innovation constraint further includes: When the change direction is the second change direction, the innovation is constrained based on the upper limit of the second innovation constraint; the second change direction represents the direction in which the sound source orientation changes towards the wide side direction, causing the absolute value of the smooth time delay state value to decrease, the wide side direction is the sound source direction perpendicular to the interval direction, and the upper limit of the first innovation constraint is greater than the upper limit of the second innovation constraint.

[0020] Optionally, in some examples, before constraining the innovation based on the direction of change and a preset upper limit for innovation constraints, the method further includes: Based on the smooth time delay projection, determine the geometric mapping sensitivity corresponding to the smooth time delay state value; Based on the preset angle change amount and the geometric mapping sensitivity, the delay change budget of the current audio frame is determined; Based on the time delay variation budget, the first direction coefficient, and the second direction coefficient, the first and second innovation constraint upper limits are determined respectively; the first direction coefficient is greater than the second direction coefficient.

[0021] Optionally, in some examples, when the sound source orientation tracking process is in the end-fire hold state and the cross-side change condition is met, the innovation constraint corresponding to the end-fire hold state is not executed, or the upper limit of the innovation constraint is increased; The smooth delay state value is updated based on the state estimation parameters corresponding to the end-fire acquisition state.

[0022] Optionally, in some examples, before determining whether the sound source location tracking process has entered the end-fire acquisition state based on the delay measurement value, smooth delay state value, and measurement confidence level of the current audio frame, the method further includes: The dual-microphone audio data of the dual-microphone array is synchronously framed to obtain the first channel audio frame and the second channel audio frame corresponding to the current audio frame; Delay estimation is performed on the first channel audio frame and the second channel audio frame to obtain the delay measurement value and the measurement confidence level; Obtain the delay measurement value corresponding to the previous audio frame of the current audio frame, and perform recursive state estimation on the delay measurement value of the previous audio frame to obtain the smooth delay state value.

[0023] Optionally, in some examples, before determining whether the sound source location tracking process has entered the end-fire acquisition state based on the delay measurement value, smooth delay state value, and measurement confidence level of the current audio frame, the method further includes: Based on the speed of sound and the distance between the two microphones, the physical delay range corresponding to the dual-microphone array is determined; When the delay measurement value exceeds the physical delay range, the delay measurement value is limited to the boundary value corresponding to the physical delay range.

[0024] Optionally, in some examples, determining the directional information of the sound source based on the updated smooth delay state value and the microphone spacing of the dual-microphone array includes: Based on the updated smooth delay state value, determine the updated smooth delay projection; Based on the geometric mapping relationship between the smooth time delay projection and the sound source location, the azimuth angle of the sound source relative to the dual-microphone array is determined.

[0025] Optionally, in some examples, determining the directional information of the sound source based on the updated smooth delay state value and the microphone spacing of the dual-microphone array further includes: When the measurement confidence level is greater than the confidence level threshold, the average signal voltage of the two microphones reaches a preset level threshold, the signal energy ratio of the two microphones meets a preset energy balance condition, and the azimuth angle is within the azimuth range corresponding to the array structure of the dual-microphone array, the sound source is determined to be in a valid azimuth and a valid flag is generated.

[0026] Optionally, in some examples, determining the directional information of the sound source based on the updated smooth delay state value and the microphone spacing of the dual-microphone array further includes: The quality score of the sound source is determined based on the measurement reliability of the current audio frame.

[0027] In another aspect of this application, an electronic device is provided, comprising: at least one processor; and a memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, perform the method according to this application.

[0028] In another aspect of this application, a non-transitory computer-readable storage medium is provided, storing computer-executable instructions, wherein the computer-executable instructions, when executed by at least one processor, perform the method according to this application.

[0029] In another aspect of this application, a computer program product is provided, including computer-executable instructions, wherein the computer-executable instructions are configured to cause the at least one processor to perform the method according to this application when executed by at least one processor.

[0030] By way of example and not limitation, the advantages provided by this application may include one or more of the following advantages and / or other advantages: 1) By combining the degree of end-fire enhancement of the current delay measurement relative to the smooth delay state, the cross-side variation of the delay projection, and the measurement reliability, distinguish between reliable end-fire changes and occasional changes caused by noise, multipath, or error correlation peaks, and improve the reliability of end-fire acquisition state judgment. 2) Different end-fire related measurement noise processing rules are adopted for normal tracking state, end-fire acquisition state and end-fire hold state to avoid using the same conservative processing for all end-fire measurements, and to balance end-fire acquisition speed and azimuth tracking stability. 3) When the sound source orientation tracking process enters the end-fire acquisition state, the influence of reliable end-fire measurement on the smooth delay state update result is improved by adjusting the state parameters, so that the smooth delay state converges to the end-fire enhancement measurement or cross-side change measurement more timely. 4) When the smooth delay state value satisfies the end-fire locking condition, it is confirmed to enter the end-fire holding state. Different upper limits of the innovation constraint are applied to the innovation in the end-fire direction and the wide side direction, so that the reliable measurement continues to drive the state to converge in the end-fire direction, while suppressing the wide side backoff caused by noise, multipath or reflection, and avoiding the problem that traditional symmetric constraints cannot take into account both end-fire acquisition capability and end-fire holding stability. 5) When the current time delay measurement and smooth time delay state characterization sound source are located on different sides of the dual-microphone array and meet the credibility conditions, skip or relax the information constraint in the end-fire holding state, and update the smooth time delay state according to the end-fire acquisition state. Avoid the excessive restriction of the real cross-side change by the wide-side backoff constraint, and improve the azimuth switching capability between the two sides of the dual-microphone array.

[0031] It should be understood that the technical problems and advantages listed above are merely examples and not limitations on this application. Furthermore, this application is not limited to technical solutions that simultaneously solve all of the above-mentioned technical problems; the technical solutions of this application can be implemented to solve one or more of the above-mentioned or other technical problems, and to provide one or more of the above-mentioned or other advantages. Attached Figure Description

[0032] Figure 1 A schematic flowchart illustrating a sound source location determination method according to an illustrative embodiment of this application is shown. Figure 2 This diagram illustrates the processing state transitions of a sound source orientation tracking process according to an illustrative embodiment of this application. Figure 3 A schematic diagram illustrating an end-fire region asymmetric information constraint according to an illustrative embodiment of this application is shown. Figure 4 The diagram illustrates the mean absolute error under different verification scenarios according to an illustrative embodiment of this application; Figure 5 This invention illustrates a sound source location determination device according to an illustrative embodiment of the present application; and Figure 6 A schematic block diagram of an electronic device according to an embodiment of this application is shown. Detailed Implementation

[0033] The solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments shown in the drawings and described below are merely illustrative and are not intended to limit the scope of this application.

[0034] In the context of this application, unless otherwise specifically stated, ordinal numbers such as "first" and "second" are used only to distinguish different elements and not to limit their order.

[0035] Unless otherwise provided, the terminology used in this application shall be interpreted in accordance with the meaning commonly understood by those skilled in the art. The terminology cited in this application is for the purpose of illustrating the technical solutions only and shall not be construed as limiting the scope of this application.

[0036] Unless otherwise expressly stated, the singular form includes the plural form, and terms such as "comprising" or "including" should be understood as open-ended expressions, including but not limited to the listed elements or steps. Those skilled in the art can reasonably interpret the terms based on the context, and the specific embodiments in the specification should not be construed as the sole limitation on the technical solutions of this application.

[0037] The technical features mentioned in this application can be combined arbitrarily among different embodiments, unless such combinations are technically unrealizable or explicitly incompatible with this application.

[0038] The method for determining the sound source location of a dual-microphone array provided in this application will now be described in detail with reference to the accompanying drawings: refer to Figure 1 The diagram shown is a flowchart illustrating a sound source location determination method according to an embodiment of this application. The execution entity of this method can be a microcontroller, digital signal processor, system-on-a-chip, personal computer, server, or other electronic device with audio data processing capabilities. For example, the device can be a microcontroller or system-on-a-chip in a voice interaction device equipped with a dual-microphone array, or it can be an audio processing device communicatively connected to the dual-microphone array. The dual-microphone array can be integrated into the device or set up independently and communicatively connected to the device. The method includes: Step 101: Based on the delay measurement value, smooth delay status value and measurement confidence of the current audio frame, determine whether the sound source orientation tracking process has entered the end-fire acquisition state.

[0039] In this embodiment of the application, the delay measurement value of the current audio frame of the dual-microphone array, the measurement reliability corresponding to the delay measurement value, and the smooth delay state value will be obtained. Based on the change of the delay measurement value relative to the smooth delay state value and the measurement reliability, it will be determined whether the current audio frame indicates a reliable end-fire acquisition requirement, and then it will be determined whether the sound source azimuth tracking process has entered the end-fire acquisition state.

[0040] In one possible implementation, this application embodiment synchronizes and frames the dual-microphone audio data of the dual-microphone array to obtain the first channel audio frame and the second channel audio frame corresponding to the current audio frame. Then, delay estimation is performed on the first channel audio frame and the second channel audio frame to obtain delay measurement values ​​and measurement confidence levels. The delay measurement values ​​corresponding to the previous audio frames of the current audio frame are also obtained, and recursive state estimation is performed on the delay measurement values ​​of the previous audio frames to obtain smooth delay state values.

[0041] For example, the current audio frame in this application represents the dual-channel audio frame corresponding to the dual-microphone audio data in the current processing period. For instance, the current audio frame may include a first-channel audio frame corresponding to the first microphone and a second-channel audio frame corresponding to the second microphone, where the first-channel and second-channel audio frames correspond to the same or mutually corresponding audio acquisition periods. The latency measurement value of the current audio frame can be the arrival time difference of the two microphones determined based on the first-channel and second-channel audio frames, used to characterize the time difference of the same sound signal arriving at the two microphones, rather than the processing latency of any single audio channel itself.

[0042] For example, the time delay measurement value in this application can be determined based on the audio signals corresponding to the two audio channels in the current audio frame, representing the time difference between the arrival of the sound signals at the two microphones. In addition, the time delay measurement value in this application can also be determined by generalized cross-correlation with phase transform (GCC-PHAT), band-limited cross-correlation, phase difference estimation, or other dual-microphone time delay estimation methods, and this application embodiment does not limit this.

[0043] For example, in this application, measurement confidence represents the reliability of the delay measurement of the current audio frame. Measurement confidence can be determined based on the characteristics of the correlation peaks in the cross-correlation sequence. For instance, this application can determine measurement confidence based on the peak-to-peak ratio between the main correlation peak and the secondary correlation peak, the ratio between the main correlation peak and the noise floor of the correlation sequence, the peak width or sharpness of the correlation peak, the consistency of the delay measurements corresponding to different frequency sub-bands, or multiple of the above parameters.

[0044] For example, the smoothed delay state value in this application can be a delay state value obtained by recursively estimating the delay measurement values ​​corresponding to one or more previous audio frames before the current audio frame, representing the historical delay state maintained by the sound source orientation tracking process before the current audio frame is updated. The recursive state estimation may include Kalman filtering, extended Kalman filtering, unscented Kalman filtering, α-β filtering, exponential recursive filtering, or other processing methods that can update the delay state based on historical states and current measurements; this application does not specifically limit this method.

[0045] In one possible implementation, when the measurement confidence level meets the preset confidence level requirement and the sound source orientation tracking process meets at least one of the end-fire enhancement condition and the translateral change condition, the embodiment of this application can determine that the sound source orientation tracking process has entered the end-fire acquisition state.

[0046] For example, the end-fire enhancement condition indicates that the sound source orientation corresponding to the delay measurement value of the current audio frame is closer to the sound source orientation corresponding to the smooth delay state value before the update. The end-fire cross-side change condition indicates that the sound source orientation corresponding to the delay measurement value and the smooth delay state value of the current audio frame are located on different sides of the dual-microphone array, respectively.

[0047] For example, the end-fire acquisition state can be understood as a processing state entered by the sound source azimuth tracking process to improve its responsiveness to the current reliable end-fire measurement. The end-fire enhancement condition indicates that the sound source azimuth corresponding to the delay measurement value of the current audio frame is closer to the end-fire direction than the sound source azimuth corresponding to the smooth delay state value before the update. The cross-side change condition indicates that the sound source azimuth corresponding to the delay measurement value and the smooth delay state value of the current audio frame are located on different sides of the dual-microphone array, indicating that the sound source has switched from one side of the dual-microphone array to the other. Thus, in this application, the end-fire enhancement condition and the cross-side change condition correspond to two different state entry conditions for triggering the end-fire acquisition state. After determining that the end-fire acquisition state has been entered, the smooth delay state value can be updated according to the state update method corresponding to the end-fire acquisition stage, thereby accurately determining the sound source azimuth.

[0048] For example, in this application, the end-fire direction is the direction of the sound source extending along the distance between the two microphones. Correspondingly, the wide-side direction is the direction of the sound source perpendicular to the distance between the two microphones. Under the far-field sound source approximation condition, the propagation path difference of the sound signal to the two microphones varies with the sound source orientation relative to the end-fire direction or the wide-side direction. The closer the sound source is to the end-fire direction, the closer the propagation path difference of the sound signal to the two microphones is to the limiting path difference determined by the distance between the two microphones, and the closer the absolute value of the arrival time difference between the audio signals corresponding to the two microphones is to the physical delay extreme value determined by the distance between the two microphones; the closer the sound source is to the wide-side direction, the closer the propagation path difference of the sound signal to the two microphones is to zero, and the closer the absolute value of the corresponding arrival time difference is to zero.

[0049] For example, refer to Figure 2The diagram shown illustrates the processing state transitions of a sound source tracking process according to an embodiment of this application. The processing states of the sound source azimuth tracking process in this application may include a normal tracking state, an end-fire acquisition state, and an end-fire holding state. The normal tracking state represents the processing state when the sound source azimuth tracking process has not entered the end-fire acquisition state and the smoothing delay state value does not meet the end-fire locking condition corresponding to the end-fire holding state. When the credible delay measurement corresponding to the current audio frame represents end-fire enhancement or cross-side change, the sound source azimuth tracking process can switch from the normal tracking state to the end-fire acquisition state. During the end-fire acquisition process, as the smooth delay state value converges to the credible delay measurement value and satisfies the end-fire locking condition, the sound source azimuth tracking process can switch to the end-fire holding state. When credible end-fire enhancement or cross-side change occurs again in the end-fire holding state, the sound source azimuth tracking process can return to the end-fire acquisition state to increase the influence of the current delay measurement value on the smooth delay state value update result. When the smooth delay state value leaves the preset delay range corresponding to the end-fire direction, the audio signal is invalid, or the conditions for resuming normal tracking are met, the sound source azimuth tracking process can switch from the end-fire holding state to the normal tracking state.

[0050] In one possible implementation, when estimating the time delay of the first channel audio frame and the second channel audio frame, this application may divide the first channel audio frame and the second channel audio frame into multiple corresponding frequency sub-bands, and determine the corresponding sub-band time delay measurement value and sub-band measurement confidence for each frequency sub-band.

[0051] For example, the first channel audio frame and the second channel audio frame can be audio frames corresponding to the two microphones in the dual-microphone array during the same audio acquisition period. Corresponding frequency sub-bands can have the same frequency range, so that this application can determine the arrival time difference of the dual microphones corresponding to the frequency sub-band based on the first channel audio signal and the second channel audio signal within the same frequency range.

[0052] In one possible implementation, this application further compares the sub-band delay measurements corresponding to each frequency sub-band. When the differences between the sub-band delay measurements meet a preset consistency condition, the sub-band delay measurements are weighted and fused based on the measurement reliability of each sub-band to obtain the delay measurement of the current audio frame. When the differences between the sub-band delay measurements of each frequency sub-band do not meet the consistency condition, the measurement reliability of the current audio frame is reduced.

[0053] For example, the consistency condition in this application can be determined based on the difference between the maximum and minimum values ​​of the delay measurements in each sub-band, the deviation of each sub-band delay measurement from the reference delay value, or the dispersion of each sub-band delay measurement. When the differences between the delay measurements in each sub-band satisfy the consistency condition, it means that the delay measurement results obtained from different frequency sub-bands are consistent with each other. This application can then determine the fusion weight corresponding to each sub-band delay measurement value based on the reliability of each sub-band measurement, and perform weighted fusion on each sub-band delay measurement value to obtain the delay measurement value of the current audio frame. The higher the reliability of the sub-band measurement, the greater the influence of the corresponding sub-band delay measurement value on the fusion result.

[0054] In one possible implementation, this application will reduce the measurement reliability of the current audio frame when the difference between the delay measurements of each sub-band does not meet the consistency condition.

[0055] For example, the discrepancy between the time delay measurements of each sub-band does not meet the consistency condition, indicating that some frequency sub-bands may be affected by noise, multipath, or reflection. In this case, instead of forcibly changing the main time delay measurement value of the current audio frame based on the time delay measurements of the sub-bands with obvious discrepancies, this application can reduce the measurement reliability of the current audio frame. The reduced measurement reliability can correspondingly increase the measurement noise, thereby reducing the impact of the current audio frame with sub-band time delay discrepancies on the smooth time delay state value update result, and thus reducing the interference of sub-band conflicts on subsequent end-fire state determination and sound source orientation tracking.

[0056] In one possible implementation, this application may first determine the cross-correlation sequence between the first channel audio frame and the second channel audio frame, and then determine multiple local correlation peaks from the cross-correlation sequence. Based on the proportional relationship between the peak value of each local correlation peak and the largest peak value among the multiple local correlation peaks, candidate correlation peaks are determined from the multiple local correlation peaks. Next, each candidate correlation peak is scored based on the difference between its candidate delay and smooth delay state value, and the target correlation peak and the delay measurement value of the current audio frame are determined based on the scoring results.

[0057] For example, different positions in the cross-correlation sequence correspond to different candidate time delays, and the peak value of a local correlation peak represents the degree of correlation between two audio signals under the corresponding candidate time delay. In acoustic environments with multipath, reflection, or reverberation, multiple local correlation peaks may appear simultaneously in the cross-correlation sequence. Therefore, determining the time delay measurement value solely based on the maximum peak value may cause unexpected jumps in the time delay measurement results. Therefore, this application first determines the maximum peak value among multiple local correlation peaks and then determines the proportional relationship between the peak value of each local correlation peak and the maximum peak value. When the proportion corresponding to a local correlation peak reaches a preset peak proportion condition, the local correlation peak is determined as a candidate correlation peak. In this way, locally correlated peaks with significantly smaller peak values ​​can be excluded, reducing the number of correlation peaks that need to be scored subsequently.

[0058] For example, for each candidate correlation peak, this application can determine the corresponding candidate delay based on the position of the candidate correlation peak in the cross-correlation sequence, and determine the continuity between the corresponding candidate correlation peak and the historical tracking state based on the difference between the candidate delay and the smoothed delay state value before the update. Then, this application can score each candidate correlation peak by combining the peak characteristics of the candidate correlation peak and the difference between the candidate delay and the smoothed delay state value. Specifically, the smaller the difference between the candidate delay and the smoothed delay state value, the higher the continuity between the corresponding candidate correlation peak and the historical tracking state; the more significant the peak value of the candidate correlation peak, the higher the probability that the corresponding candidate correlation peak reflects an effective delay measurement.

[0059] For example, this application determines the target correlation peak from multiple candidate correlation peaks based on the scoring results. For instance, the candidate correlation peak with the highest score is determined as the target correlation peak, and the delay measurement of the current audio frame is determined based on the position of the target correlation peak in the cross-correlation sequence. Thus, by comprehensively considering the peak characteristics and delay continuity of the correlation peaks, this application can reduce the possibility of short-time competing peaks generated by multipath propagation or reflections being mistakenly selected as the target correlation peak.

[0060] In one possible implementation, during normal tracking, scoring based on the difference between candidate delays and smoothed delay state values ​​can suppress erroneous correlation peaks that significantly deviate from historical tracking states. However, when the smoothed delay state value still corresponds to the source azimuth near the wide side, while the current source is approaching the end-fire direction, the candidate delay corresponding to the true end-fire correlation peak may differ significantly from the smoothed delay state value. If a large scoring weight is still applied to this difference, the end-fire correlation peak may be excluded due to deviation from historical wide-side states, thus hindering the source azimuth tracking process from obtaining the end-fire direction.

[0061] Therefore, in one possible implementation, when the sound source orientation tracking process enters the end-fire acquisition state, this application can skip the process of scoring candidate correlation peaks based on the difference between candidate delay and smooth delay state values, or reduce the weight of this difference in the candidate correlation peak scoring. Skipping the candidate correlation peak scoring process can mean skipping the continuous scoring based on the difference between candidate delay and smooth delay state values, without canceling the screening process performed based on the peak value, peak ratio, or other peak characteristics of the candidate correlation peaks. This application can still determine the target correlation peak based on the peak characteristics of each candidate correlation peak, and determine the delay measurement value of the current audio frame based on the target correlation peak.

[0062] In summary, this application reduces the measurement reliability when subband delay diverges, thereby mitigating the impact of local band interference on state updates. By combining peak characteristics and historical delay continuity to select the target correlation peak, it improves the stability of delay measurements in multipath environments. Furthermore, by skipping or weakening historical continuity constraints in the end-fire acquisition state, it avoids historical wide-side states from hindering the selection of effective end-fire correlation peaks, thus balancing the stability of delay measurements with the ability to acquire end-fire directions. It should be noted that the above processing is an optional implementation of the delay estimation process and does not constitute a necessary limitation on the end-fire acquisition and end-fire holding scheme of this application.

[0063] In one possible implementation, in order to facilitate a unified determination of the degree of proximity of the sound source to the end-fire direction under different dual-microphone spacing and sound speed conditions, embodiments of this application may also normalize the time delay measurement value and smooth time delay state value of the current audio frame based on the ratio of sound speed to dual-microphone spacing, respectively, to obtain the time delay measurement projection and smooth time delay projection.

[0064] For example, the time delay measurement projection and the smoothed time delay projection in this application can be determined according to the following formulas:

[0065] in, Indicates the projection of time delay measurement. Indicates smooth time-delay projection; Indicates the speed of sound. This indicates the spacing between the two microphones; This represents the latency measurement value currently awaiting state update. This represents the smoothed latency state value before the update. This is without any further preprocessing of the latency measurement values. It can be the raw time delay measurement value directly determined from the current audio frame; or the time delay measurement value after physical boundary constraints, median filtering, or other measurement preprocessing. It can be a delay measurement value used for state update after appropriate preprocessing.

[0066] Thus, this application can limit the smooth time delay projection to a preset projection range, for example, limiting it to a range from -1 to 1, to avoid calculation errors causing the smooth time delay projection to exceed the domain of the inverse cosine function. The time delay measurement projection and the smooth time delay projection can both lie within a normalized projection range from -1 to +1. The sign of the time delay projection is used to distinguish the side of the sound source relative to the dual-microphone array, and the absolute value of the time delay projection represents the degree to which the sound source approaches the end-fire direction. When the absolute value of the time delay projection is close to one, it indicates that the sound source azimuth corresponding to the corresponding time delay is close to the end-fire direction; when the time delay projection is close to zero, it indicates that the sound source azimuth corresponding to the corresponding time delay is close to the wide-side direction. By using the normalized time delay projection for state judgment, the influence of different dual-microphone spacing or different sound velocity parameters on the end-fire judgment threshold setting can be reduced, giving the end-fire acquisition conditions a relatively uniform numerical scale.

[0067] It is worth mentioning that the specific positive and negative directions of the time delay projection symbol in this application are related to the channel order of the two microphones and the time delay definition method. This application does not restrict which specific positive and negative directions of time delay are used, as long as a consistent definition is used in the time delay estimation, smooth time delay state update and sound source location determination processes. The physical time delay range represents the actual arrival time difference range that can be formed based on the geometry of the dual-microphone array.

[0068] In one possible implementation, the embodiments of this application will also determine the physical delay range corresponding to the dual-microphone array based on the sound speed and the distance between the two microphones, and when the delay measurement value exceeds the physical delay range, the delay measurement value will be limited to the boundary value corresponding to the physical delay range.

[0069] For example, due to the discreteness of audio sampling, environmental noise, multipath propagation, sound reflection, or correlation peak selection error, the original time delay measurement may temporarily exceed the physical time delay range determined based on the microphone spacing and sound speed. This original time delay measurement exceeding the physical time delay range cannot directly represent the actual propagation time difference that the microphone array can generate. Therefore, this application will perform physical boundary processing on the original time delay measurement before using it to determine the end-fire acquisition state or update the smoothed time delay state value.

[0070] For example, suppose the distance between the two microphones is d, the speed of sound is c, and the time difference between the arrival of the sound source signal at the two microphones is τ. Since the difference in the propagation path of the sound source signal to the two microphones is physically no more than the distance d between the two microphones, the absolute value of the maximum time delay corresponding to this dual-microphone array can be expressed as: The corresponding physical delay range can be expressed as:

[0071] Thus, if the original delay If the value exceeds the above range, it is restricted to the corresponding boundary value, thereby reducing the impact of superphysical peaks caused by sampling discreteness, noise, or multipath on subsequent state judgments. This represents the upper boundary of the physical time delay range. This represents the lower boundary of the physical delay range.

[0072] For example, the microphone pitch represents the distance between the acoustic centers of the two microphones in the dual-microphone array, and the speed of sound represents the speed of sound propagation in the current propagation medium of the dual-microphone array. In this embodiment, the speed of sound can be a pre-configured fixed value or a speed of sound determined according to preset environmental parameters. This embodiment does not limit the specific values ​​of the speed of sound and the microphone pitch, as long as they can be used to determine the allowable physical delay range of the dual-microphone array under the current application conditions.

[0073] In summary, after the above physical boundary processing, this application can further refine the processed time delay measurement value. Used to determine the projection of the time delay measurement. Due to the time delay measurement value... It has been restricted to being by to Within the physical time delay range, therefore, the time delay measurement projection... Correspondingly located in the area of to Within the projected range, delay measurements exceeding the physical range are avoided from directly participating in the determination of end-fire enhancement conditions or trans-side change conditions.

[0074] In one possible implementation, the physical boundary processing in the embodiments of this application does not necessarily have to adopt the above-mentioned hard limit. It can also adopt one or more of the following processing methods: reducing the confidence of delays that exceed the range, using a soft saturation function for delays that exceed the range, increasing the measurement noise according to the amount of excess, and directly rejecting the measurement of the frame.

[0075] For example, embodiments of this application may first apply hard limiting or soft saturation processing to the original delay measurement value that exceeds the physical delay range, and then reduce the measurement reliability or increase the measurement noise based on the amount of the original delay measurement value exceeding the limit; alternatively, boundary limiting may be applied to the original delay measurement value when the amount of the exceedance is less than a preset exceedance threshold, and the delay measurement value of the current audio frame may be rejected when the amount of the exceedance is greater than or equal to the preset exceedance threshold. The preset exceedance threshold is used to distinguish between slightly excessive measurements that can continue to be used through boundary processing and obviously abnormal measurements that are unsuitable for continuing to participate in state judgment or state update. Its specific value can be determined based on the microphone spacing, sampling rate, delay estimation resolution, and predetermined acoustic environment.

[0076] In one possible implementation, this application can determine whether the current delay measurement result indicates that the sound source is approaching the end-fire direction further based on the change of the delay measurement value of the current audio frame relative to the smooth delay state value.

[0077] For example, this application addresses the current measurement z to be updated and the current smoothing state. Calculate the time delay measurement projection respectively and smooth time delay projection The difference between the corresponding absolute values ​​is expressed as

[0078] Therefore, in one possible implementation, if the measurement confidence level meets a preset confidence level requirement, and at least one of the end-firing enhancement condition or the translateral change condition is met, then the sound source azimuth tracking process is determined to have entered the end-firing acquisition state. Wherein: 1. End-fire enhancement conditions: Compare Large preset increment value, and Greater than the threshold for end-shot acquisition; 2. Translateral variation conditions: and The signs are opposite, and Greater than the cross-lateral change threshold; 3. Credibility condition: The credibility of the current relevant peak is greater than the credibility threshold.

[0079] In one possible implementation, for end-fire enhancement conditions, this embodiment of the application will determine that the sound source azimuth tracking process meets the end-fire enhancement conditions and enter the end-fire acquisition state when the measurement confidence is greater than a preset confidence threshold, the difference between the absolute value of the delay measurement projection and the absolute value of the smooth delay projection reaches a preset increment value, and the absolute value of the delay measurement projection is greater than a preset end-fire acquisition threshold.

[0080] For example, in this application, the confidence threshold is used to distinguish between reliable delay measurements and unreliable delay measurements that may be caused by noise, reverberation, multipath, or erroneous correlation peaks. When the measurement confidence is greater than the preset confidence threshold, it indicates that the delay measurement value of the current audio frame has a level of reliability sufficient to participate in end-fire acquisition judgment and state update.

[0081] For example, in this application, a preset increment value is used to limit the minimum degree of change in the current delay measurement relative to the existing smooth delay state that leads further closer to the end-fire direction, avoiding frequent triggering of end-fire enhancement conditions due to minute delay projection changes caused by measurement noise. A preset end-fire acquisition threshold is used to limit the current delay measurement itself to a predetermined end-fire proximity level, preventing misjudgment that the end-fire enhancement condition is met when the current measurement, although increased relative to the smooth delay state, is still near the wide side. For example, the end-fire acquisition threshold can be set in the range of 0.6 to 0.9, and the preset increment value can be set in the range of 0.05 to 0.30. The confidence threshold, end-fire acquisition threshold, and preset increment value can be determined based on the microphone spacing, audio sampling rate, resolution of the delay estimation algorithm, noise and reverberation levels in the acoustic environment, and the product's requirements for response speed and tracking stability. For example, in applications requiring increased end-fire acquisition sensitivity, the end-fire acquisition threshold or preset increment value can be appropriately lowered; in applications with strong multipath interference or noise, the confidence threshold, end-fire acquisition threshold, or preset increment value can be appropriately increased to reduce the probability of occasional erroneous measurements triggering end-fire enhancement conditions. Clearly, the above value range is only used to illustrate one possible parameter setting method and does not constitute a limitation on the scope of protection of this application.

[0082] In one possible implementation, this application embodiment will determine that the sound source orientation tracking process meets the cross-side change condition and enter the end-fire acquisition state when the measurement confidence is greater than a preset confidence threshold, the signs of the time delay measurement projection and the smooth time delay projection are opposite, and the absolute value of the time delay measurement projection is greater than a preset cross-side change threshold.

[0083] In this embodiment, the signs of the delay measurement projection and the smooth delay projection are opposite, representing that the delay measurement value and the smooth delay state value of the current audio frame point to different sides of the dual-microphone array, respectively. If the absolute value of the delay measurement projection is greater than a preset cross-side change threshold, it indicates that the current delay measurement has deviated from the zero-delay region near the wide side, thereby reducing misjudgments caused by sign changes due to measurement disturbances near the wide side. This application, combined with a measurement confidence level greater than a preset confidence threshold, can determine that the delay measurement value of the current audio frame can reliably characterize the sound source switching from one side of the dual-microphone array to the other.

[0084] For example, the translateral variation threshold can be set in the range of 0.5 to 0.9, and the specific value can be determined according to the microphone spacing, sampling rate, acoustic environment and translateral response requirements.

[0085] In one possible implementation, the embodiments of this application will also determine that the sound source azimuth tracking process enters the end-fire holding state when the smooth delay state value meets the preset end-fire locking condition.

[0086] For example, in this application, the end-fire hold state represents that the smooth delay state value is within the preset delay range corresponding to the end-fire direction. The sound source orientation tracking process needs to reduce the possibility that the smooth delay state value will unexpectedly regress from the end-fire direction to the wide side direction due to factors such as noise, sound reflection, multipath propagation, or erroneous correlation peaks while continuing to respond to the reliable delay measurement value of the current audio frame.

[0087] In one possible implementation, the end-fire locking condition in this application embodiment includes the absolute value of the smoothed delay projection being greater than a preset end-fire holding threshold, and includes: The state uncertainty of the smooth delay state value is less than the preset locking threshold.

[0088] The absolute value of the smooth delay projection is greater than the preset end-shooting hold threshold.

[0089] For example, if the state uncertainty of the smoothed delay state value is less than a preset locking threshold, it indicates that the recursive state estimation process has reached a preset convergence level, and the state uncertainty represents the degree of uncertainty of the recursive state estimation process regarding the current smoothed delay state value. For instance, when the recursive state estimation uses Kalman filtering, the state uncertainty can be the error covariance corresponding to the smoothed delay state value. The smaller the state uncertainty, the more it indicates that the smoothed delay state value has gradually converged based on the delay measurement of consecutive audio frames, and the current tracking state is relatively stable. In this application, the preset locking threshold can be determined based on the allowable delay estimation error, the allowable degree of sound source azimuth fluctuation, the distance between the two microphones, the audio sampling rate, and the expected end-fire locking speed. When the state uncertainty is less than the preset locking threshold, this application considers that the recursive state estimation process has reached a stable level suitable for performing end-fire hold processing.

[0090] For example, if the absolute value of the smoothed delay projection is greater than a preset end-fire holding threshold, it means that the smoothed delay state value is within the preset delay range corresponding to the end-fire direction. When both of the above conditions are met, it can be determined that the smoothed delay state value meets the end-fire locking condition, reducing the possibility of prematurely entering the end-fire holding state due to temporary end-fire measurement. This application can determine that the smoothed delay state value meets the end-fire locking condition based on any one of the above end-fire locking conditions, or it can determine that the smoothed delay state value meets the end-fire locking condition based on a combination of the two conditions.

[0091] In one possible implementation, before determining the state uncertainty of the smoothed delay state value, this application can first confirm whether the recursive state estimation process has been initialized and whether the current smoothed delay state value is valid. A valid smoothed delay state value means that the currently stored smoothed delay state value can be used for the state update of the current audio frame; therefore, this validity determination can be used to reliably determine the state uncertainty. This application can mark the corresponding smoothed delay state value as valid after completing state initialization based on one or more delay measurement values ​​whose measurement confidence meets the confidence requirements. When the recursive state estimation process has not yet been initialized, the smoothed delay state value does not exist, the smoothed delay state value exceeds the physical delay range corresponding to the dual-microphone array, or the smoothed delay state value is marked as unusable for the current state update, it is determined that there is no valid smoothed delay state value. Thus, by determining whether a valid smoothed delay state value exists, this application can avoid performing end-fire hold processing before a usable delay state is established in the sound source azimuth tracking process.

[0092] In one possible implementation, the state uncertainty can be determined based on the recursive state estimation results of consecutive audio frames. To improve the reliability of the state uncertainty, this application can also accumulate at least a preset number of effective delay measurements and recursively update the smooth delay state value and its state uncertainty based on the effective delay measurements. Here, an effective delay measurement represents a delay measurement value whose measurement reliability meets a preset reliability requirement. This application can count the already obtained effective delay measurements and determine that the corresponding measurement quantity condition is met when the cumulative number of effective delay measurements reaches a preset number. The preset number can be determined based on the audio frame length, frame shift, expected lock time, delay measurement stability, and acoustic environment. Effective delay measurements can be continuously obtained effective delay measurements or accumulated effective delay measurements within a preset statistical time period. Thus, by using continuous effective delay measurements, this application can further confirm the temporal continuity of the current end-fire measurement; by using accumulated effective delay measurements within a preset statistical time period, it can reduce the possibility of invalid audio frames interrupting the end-fire lock judgment.

[0093] In summary, by combining the above conditions, the reliability of end-fire hold-up determination can be improved, and the possibility of the sound source azimuth tracking process prematurely entering the end-fire hold-up state based on a single audio frame or temporary measurement results can be reduced. It should be noted that the above combination of conditions is not intended to limit other implementation methods. In other implementation methods, the satisfaction of the end-fire lock-up condition can also be determined based on one or a combination of the above conditions.

[0094] In one possible implementation, since the time delay of the dual-microphone array and the azimuth angle of the sound source are non-linearly mapped, the same amount of time delay change may correspond to different amounts of azimuth angle change at different sound source locations. Especially in the end-fire region, as the absolute value of the smoothed time delay projection increases, the degree of sound source azimuth angle change corresponding to a unit time delay change usually increases. Therefore, if a fixed upper limit of time delay constraint is applied to each sound source location, a small time delay disturbance in the end-fire region may still be mapped to a large azimuth angle change. Considering the above problems, this embodiment of the application, after determining that the sound source azimuth tracking process has entered the end-fire hold state, determines the information between the time delay measurement value and the smoothed time delay state value before the update, as well as the direction of change corresponding to the information. Based on the direction of change and the upper limit of the information constraint, the information is constrained, thereby updating the smoothed time delay state value based on the constrained information. Thus, by constraining the information participating in the state update in the end-fire hold state, the excessive pulling effect of abnormal time delay measurement values ​​in a single audio frame on the smoothed time delay state value can be limited.

[0095] For example, in the embodiments of this application, the new information represents the difference between the delay measurement value and the smooth delay state value before the update, and the direction of change represents the trend of the absolute value of the smooth delay state value when the smooth delay state value is updated based on the new information.

[0096] For example, in this application, the innovation represents the difference between the latency measurement of the current audio frame and the smooth latency state value before the update, as shown in the following formula:

[0097] in, The new information represents the state correction amount proposed by the delay measurement result of the current audio frame relative to the existing smooth delay state value; This is the latency measurement currently to be used for state update. This represents the smoothed delay state value before the update.

[0098] Without preprocessing the delay measurement values This can be the raw delay measurement value of the current audio frame. After the delay measurement value has undergone physical boundary constraints, median filtering, or other measurement preprocessing, It can be a delay measurement value used for state update after appropriate preprocessing.

[0099] In one possible implementation, the direction of change corresponding to the new information represents the trend of change in the absolute value of the smooth delay state value when updating the smooth delay state value according to the new information. It should be noted that the direction of change does not directly indicate that the sound source has objectively moved along the corresponding direction, but rather represents the state update trend indicated by the current delay measurement relative to the smooth delay state value. Considering that the current delay measurement may be affected by noise, multipath propagation, or erroneous correlation peaks, this embodiment of the application controls the updating effect of the current delay measurement on the smooth delay state value by applying different upper limits to the new information corresponding to different directions of change.

[0100] In one possible implementation, this application embodiment constrains the innovation based on a first innovation constraint upper limit when the direction of change is a first direction of change. Here, the first direction of change represents the direction in which the sound source orientation changes towards the end-emitting direction, causing the absolute value of the smooth time delay state value to increase.

[0101] In one possible implementation, this application embodiment constrains the innovation based on a second innovation constraint upper limit when the direction of change is a second direction of change. Here, the second direction of change represents the direction in which the sound source orientation changes towards the wide side, causing the absolute value of the smoothed delay state value to decrease, and the first innovation constraint upper limit is greater than the second innovation constraint upper limit.

[0102] For example, when the current latency measurement value and the smoothed latency state value correspond to the same side of the dual-microphone array, if the absolute value of the current latency measurement value is greater than the absolute value of the smoothed latency state value, the news indicates that the absolute value of the updated smoothed latency state value increases, and the corresponding change direction can be determined as the first change direction; if the absolute value of the current latency measurement value is less than the absolute value of the smoothed latency state value, the news indicates that the absolute value of the updated smoothed latency state value decreases, and the corresponding change direction can be determined as the second change direction. When using the product of the news and the smoothed latency state value for judgment, if no cross-side change occurs, This allows us to determine the direction of the first change corresponding to the new information; if This allows us to determine the second direction of change corresponding to the new information.

[0103] For example, the first direction of change is the direction that increases the absolute value of the smoothed delay state value, corresponding to the change in the source azimuth tracking result towards the end-fire direction. If the absolute value of the innovation is not greater than the upper limit of the first innovation constraint, the innovation can be retained; if the absolute value of the innovation is greater than the upper limit of the first innovation constraint, the absolute value of the innovation can be restricted to the upper limit of the first innovation constraint, while keeping the sign of the innovation unchanged. The innovation constrained by the upper limit of the first innovation constraint can continue to be used to update the smoothed delay state value.

[0104] For example, the second change direction is the direction that reduces the absolute value of the smoothed delay state value, corresponding to the change in the sound source azimuth tracking result towards the wide side. If the absolute value of the innovation is not greater than the upper limit of the second innovation constraint, the innovation can be retained; if the absolute value of the innovation is greater than the upper limit of the second innovation constraint, the absolute value of the innovation can be restricted to the upper limit of the second innovation constraint, while keeping the sign of the innovation unchanged. At the same time, the upper limit of the first innovation constraint is greater than the upper limit of the second innovation constraint, allowing relatively large end-fire state updates in the end-fire holding state, and imposing stricter constraints on the state updates towards the wide side.

[0105] For example, the first and second innovation constraint upper limits can be expressed as follows:

[0106] in, The upper limit of the first information constraint, This is the upper limit of the second information constraint. The first direction coefficient, The first direction coefficient is greater than the second direction coefficient.

[0107] Since both the first and second innovation constraint upper limits are determined based on the delay variation budget of the current audio frame, both innovation constraint upper limits can dynamically change with the geometric mapping sensitivity corresponding to the smooth delay state value. Because the first direction coefficient is greater than the second direction coefficient, the first innovation constraint upper limit is greater than the second innovation constraint upper limit, thus forming an asymmetric innovation constraint.

[0108] For example, the first directional coefficient can be set in the range of 1.0 to 5.0, and the second directional coefficient can be set in the range of 0.1 to 1.0. The specific values ​​of the first and second directional coefficients can be determined based on the microphone spacing, sampling rate, delay estimation accuracy, acoustic environment, desired end-fire acquisition speed, and end-fire holding stability. A relatively large first directional coefficient will allow the reliable delay measurement to continue pushing the smooth delay state value towards an increasing absolute delay value; a relatively small second directional coefficient will limit noise, sound reflections, multipath propagation, or erroneous correlation peaks from pulling the smooth delay state value towards a decreasing absolute delay value. Thus, during the end-fire holding phase, the tracker allows the reliable measurement to continue converging towards the end-fire while limiting wide-side backoff caused by noise or reflections. It is worth noting that the above numerical ranges are only used to illustrate one possible parameter setting method and do not constitute a limitation on the scope of protection of this application.

[0109] In one possible implementation, this application embodiment will determine the geometric mapping sensitivity corresponding to the smooth delay state value based on smooth delay projection, and determine the delay change budget of the current audio frame based on a preset angle change amount and geometric mapping sensitivity. Then, based on the delay change budget, the first direction coefficient, and the second direction coefficient, the first innovation constraint upper limit and the second innovation constraint upper limit will be determined respectively. The first direction coefficient is greater than the second direction coefficient.

[0110] For example, this application can determine the geometric mapping sensitivity corresponding to the smooth time delay state value based on the smooth time delay projection. The geometric mapping sensitivity represents the degree of change in the sound source azimuth angle corresponding to a unit change in the smooth time delay projection under the current smooth time delay state value. When the sound source azimuth angle and the smooth time delay projection satisfy an inverse cosine geometric mapping relationship, the geometric sensitivity coefficient can be determined based on the smooth time delay projection. As shown below:

[0111] in, Represents the geometric sensitivity coefficient. Indicates smooth time-delay projection, This indicates the preset upper limit of sensitivity.

[0112] As the absolute value of the smoothed time-delay projection approaches one, the unrestricted geometric sensitivity coefficient will gradually increase. Therefore, this application employs a sensitivity upper limit. Constraining the geometric sensitivity coefficient can prevent the absolute value of the smoothed delay projection from becoming too large or diverging when they are close to each other. The upper limit of sensitivity can be determined based on the numerical processing accuracy, end-fire holding range, microphone spacing, and allowable delay update amplitude.

[0113] For example, the angle change amount can be preset in this application. Based on this angle change and geometric mapping sensitivity, the latency budget for the current audio frame is determined. This angle change can be used as the angular reference for the allowable change in the sound source azimuth angle in a single state update, also known as the basic angle budget. The time delay change budget represents the time delay change corresponding to this angle change under the geometric mapping sensitivity corresponding to the current smooth time delay state value, as shown in the following formula:

[0114] in, Represents the budget for time delay variations. This represents the preset angle change. Represents the speed of sound. Represents the spacing between the two microphones. This represents the geometric sensitivity coefficient.

[0115] It should be noted that the angular change and geometric sensitivity should use matching angular units. For example, when the geometric sensitivity is determined in radians, the angular change can be converted to radians before being used in the calculation.

[0116] In summary, the time delay variation budget is not the final output of the sound source azimuth tracking process, but rather a common fundamental scale used to determine the upper limits of the first and second innovation constraints. The larger the absolute value of the smoothed time delay projection, the closer the smoothed time delay state value is to the end-fire direction, and the greater the corresponding geometric mapping sensitivity. Furthermore, with the preset angle change remaining constant, the time delay variation budget calculated from this geometric mapping sensitivity is smaller. Thus, in this application, the upper limit of the innovation constraint in the end-fire region can adaptively change with the current geometric mapping sensitivity, giving the innovation constraint in the time delay domain a relatively clear meaning regarding azimuth angle changes.

[0117] In one possible implementation, refer to Figure 3 The diagram illustrates an asymmetric constraint on the end-fire region according to an embodiment of this application. The horizontal axis represents the direction of change corresponding to the innovation, and the vertical axis represents the upper limit of the innovation constraint. The current end-fire state in the diagram corresponds to the smooth delay state value before the update. After entering the end-fire holding state during the sound source azimuth tracking process, if the innovation corresponds to the first direction of change that increases the absolute value of the smooth delay state value (i.e., the sound source azimuth changes towards the end-fire direction), a larger upper limit of the first innovation constraint is used to constrain the innovation, allowing the reliable delay measurement to continue to drive the smooth delay state value to converge towards the end-fire direction. If the innovation corresponds to the second direction of change that decreases the absolute value of the smooth delay state value (i.e., the sound source azimuth changes towards the wide side direction), a smaller upper limit of the second innovation constraint is used to constrain the innovation, limiting the smooth delay state value caused by noise, reflection, or erroneous correlation peaks from regressing towards the wide side direction.

[0118] In one possible implementation, before determining the direction of change of the innovation in the end-fire hold state, this application may first determine whether the current audio frame indicates a cross-side change. If the current delay measurement value and the smooth delay state value have opposite signs, and the absolute value of the delay measurement projection of the current delay measurement value and the measurement confidence satisfy the cross-side change condition, then the innovation may indicate that the sound source has switched from one side of the dual-microphone array to the other side, rather than a normal back-off to the wide side. In this case, if the entire innovation is still constrained by a smaller upper limit of the second innovation constraint according to the second change direction, the smooth delay state value may not be able to cross the wide side direction and converge to the other side of the dual-microphone array.

[0119] Therefore, in this embodiment of the application, when the sound source orientation tracking process is in the end-fire hold state and the cross-side change condition is met, the innovation constraint corresponding to the end-fire hold state is not executed, or the upper limit of the innovation constraint is increased, and the smooth delay state value is updated based on the state estimation parameters corresponding to the end-fire acquisition state, so as to avoid the wide-side back-off restriction in the end-fire hold stage from hindering the reliable cross-side change.

[0120] For example, increasing the upper limit of the innovation constraint in this application may include increasing the upper limit of the second innovation constraint, adopting the upper limit of the independent innovation constraint corresponding to the cross-side change condition, or temporarily adopting the upper limit of the first innovation constraint. In this way, by skipping or relaxing the innovation constraint corresponding to the end-fire hold state, it is possible to avoid effective cross-side changes being mistaken for ordinary wide-side backoff, thereby taking into account both the stability of the end-fire azimuth and the azimuth switching capability between the two sides of the dual-microphone array.

[0121] In one possible implementation, when the current audio frame does not meet the cross-side change condition, this application can continue to determine whether the current innovation corresponds to the first change direction or the second change direction, and perform end-fire holding processing using the corresponding innovation constraint upper limit. By first determining the cross-side change condition and then determining the ordinary first or second change direction, it is possible to distinguish between the effective change of the sound source switching from one end-fire side to another and the undesired backoff to the wide side caused by noise or reflection, avoiding both being processed by the same smaller innovation constraint upper limit.

[0122] Step 102: When the sound source orientation tracking process enters the end-fire acquisition state, adjust the state estimation parameters and update the smooth time delay state value based on the adjusted state estimation parameters.

[0123] In this embodiment, when the sound source azimuth tracking process has not entered the end-fire acquisition state, ordinary tracking parameters can be used to update the smoothed delay state value to reduce state fluctuations caused by noise, reverberation, or erroneous correlation peaks. When it is determined that the sound source azimuth tracking process has entered the end-fire acquisition state, the application can adjust the state estimation parameters to control the influence of the delay measurement value on the smoothed delay state value update result, so that the smoothed delay state value can converge to the reliable end-fire delay measurement value more quickly.

[0124] In one possible implementation, the adjustment of the state estimation parameters in the embodiments of this application includes increasing the degree of influence of the delay measurement value on the smoothing of the delay state value update.

[0125] For example, the state estimation parameters in this application may include one or more of the following: state uncertainty corresponding to the smoothed delay state value, process noise corresponding to the state prediction process, measurement noise corresponding to the current delay measurement value, state update gain, measurement weight corresponding to the current delay measurement value, and noise amplification coefficient set for end-fire measurement. The purpose of adjusting the state estimation parameters is not to unconditionally accept the delay measurement value of the current audio frame, but to improve the update effect of the current delay measurement value relative to the existing smoothed delay state value, provided that the current delay measurement value meets the confidence requirements and reflects a clear end-fire enhancement trend.

[0126] In one possible implementation, increasing the impact of the delay measurement on the smoothing of the delay state value update includes one or more combinations of the following operations: Improve the state uncertainty corresponding to the smoothed time delay state value; Increase the process noise corresponding to the state prediction process with increased smoothing delay state values; Reduce the measurement noise corresponding to the time delay measurement value; Increase the state update gain corresponding to the delay measurement value; Increase the measurement weight corresponding to the delay measurement value; Reduce the noise amplification factor set for end-fire measurements.

[0127] For example, when Kalman filtering is used for recursive state estimation, the state estimation parameters may include one or more of state uncertainty, process noise, measurement noise, and state update gain. In other recursive state estimation methods, this application can also increase the influence of the current time delay measurement on the state update result by increasing the measurement weight corresponding to the current time delay measurement, decreasing the state weight corresponding to the historical smooth time delay state value, resetting some state parameters, or using other equivalent methods. Thus, when the current time delay measurement indicates that the sound source is further approaching the end-fire direction, or indicates that the sound source has switched from one side of the dual-microphone array to the other, this application can reduce the excessive suppression of the current time delay measurement by the historical smooth time delay state value, so that the updated smooth time delay state value reflects the change in the current sound source location in a timely manner.

[0128] For example, in an implementation where recursive state estimation employs Kalman filtering, the measurement noise may include the measurement noise covariance corresponding to the delay measurement of the current audio frame. Measure the noise covariance This represents the degree of estimation of the measurement error contained in the current delay measurement by this application. Measurement noise covariance. The larger the value, the higher the uncertainty of the current delay measurement, and the lower the influence of the current delay measurement on the smooth delay state value update result; measurement noise covariance The smaller the value, the lower the uncertainty of the current delay measurement, and the higher the influence of the current delay measurement on the result of smoothing the delay state value update.

[0129] In one possible implementation, when the sound source azimuth tracking process is in a normal tracking state, the basic measurement noise can be determined based on the measurement reliability of the current audio frame, and conventional end-fire correlation amplification processing can be applied to the basic measurement noise based on the absolute value of the time delay measurement projection. End-fire correlation noise amplification processing in normal tracking state is used to reduce the possibility of significant changes in the sound source azimuth result caused by occasional abnormal time delay measurements in the end-fire region. However, the end-fire correlation noise processing in this application does not apply the same amplification or reduction processing to all time delay measurements close to the end-fire direction. Instead, a corresponding measurement noise adjustment strategy is selected based on the end-fire state and measurement change direction of the sound source azimuth tracking process.

[0130] For example, when the sound source orientation tracking process enters the end-fire acquisition state, this application may not apply end-fire correlation amplification processing to the basic measurement noise corresponding to the time delay measurement value of the current audio frame, or reduce or weaken the amplification degree of end-fire correlation noise amplification processing, so that the current time delay measurement value has a greater influence on the smooth time delay state value update result, so that reliable measurement can pull the state.

[0131] For example, when the smoothed delay state value meets the preset end-fire locking condition and the sound source azimuth tracking process enters the end-fire holding state, this application can retain the end-fire related noise amplification processing, but make the end-fire related noise amplification degree in the end-fire holding state less than the end-fire related noise amplification degree in the normal tracking state. In this way, a relatively weak end-fire related noise amplification factor can be used in the end-fire holding state of this application, on the one hand reducing the influence of noise, multipath propagation, sound reflection or erroneous correlation peaks in the end-fire area on the smoothed delay state value, and on the other hand avoiding excessive measurement noise that would make the smoothed delay state value completely unable to respond to continuous and reliable end-fire measurement, so as to suppress abnormal measurements at the end-fire.

[0132] In one possible implementation, when the sound source azimuth tracking process is determined to meet the end-fire enhancement condition and enters the end-fire acquisition state, this application can increase the state uncertainty corresponding to the smooth delay state value to no less than a preset lower uncertainty limit. If the sound source azimuth tracking process has been stable in the wide-side region for a long time, the state uncertainty may have decreased to a low level, making the predicted state uncertainty relatively small compared to the measurement noise, thus resulting in a small state update gain. Therefore, by increasing the state uncertainty to no less than the preset lower uncertainty limit, this application can increase the predicted state uncertainty and correspondingly increase the state update gain corresponding to the current audio frame, enabling the current reliable delay measurement value to have a greater update effect on the smooth delay state value.

[0133] In one possible implementation, when the sound source orientation tracking process satisfies the end-fire enhancement condition and enters the end-fire acquisition state, this application can further reduce the measurement noise corresponding to the time delay measurement value of the current audio frame, or cancel or reduce the additional noise increase set for end-fire measurement. For ordinary end-fire measurement, the measurement noise can be appropriately increased as the absolute value of the time delay projection increases to suppress state jumps caused by occasional error correlation peaks in the end-fire region. However, if the current audio frame already simultaneously satisfies the measurement confidence condition, the end-fire acquisition threshold condition, and the end-fire enhancement increment condition, the aforementioned end-fire related noise increase can be canceled or weakened, so that the current reliable end-fire measurement is no longer subject to the same degree of conservative weighting as ordinary end-fire measurement.

[0134] In one possible implementation, this application can simultaneously increase the state uncertainty corresponding to the smooth delay state value and reduce the measurement noise corresponding to the current delay measurement value. According to the above-described calculation relationship of state update gain, increasing the predicted state uncertainty or reducing the measurement noise can both increase the state update gain. Performing both processes simultaneously can further enhance the updating effect of the current reliable end-fire measurement on the smooth delay state value. In other implementations, the same or similar state update effect as the above parameter adjustments can be achieved by increasing process noise, directly increasing the state update gain, increasing the measurement weight corresponding to the current delay measurement value, or decreasing the retention weight corresponding to the historical smooth delay state value.

[0135] In one possible implementation, if multiple consecutive current audio frames satisfy the end-fire enhancement condition, this application can maintain the end-fire acquisition state across the corresponding audio frames and update the smooth delay state value frame by frame using the state estimation parameters corresponding to the end-fire acquisition state. As the smooth delay state value gradually approaches the reliable end-fire measurement value corresponding to the consecutive audio frames, the difference between the delay measurement projection and the smooth delay projection can gradually decrease. When the smooth delay state value reaches the preset end-fire locking condition, this application can end or reduce the response enhancement processing in the end-fire acquisition state and enter the end-fire holding state according to subsequent implementations. In this way, over-adjustment of the smooth delay state value based on only a single audio frame is avoided, and premature termination of the end-fire acquisition state can also prevent the smooth delay state value from remaining in the wide-side region.

[0136] It should be noted that during the sound source azimuth tracking process, when entering the end-fire acquisition state due to meeting the end-fire enhancement condition, whether to increase the state uncertainty, increase process noise, reduce measurement noise, increase state update gain, increase measurement weight, or decrease the end-fire related noise amplification coefficient can be determined based on the specific recursive state estimation method and equipment processing requirements. The above processing can be performed individually or in combination, as long as it can increase the influence of the current time delay measurement value on the smooth time delay state value update result when the current time delay measurement value meets the reliable end-fire enhancement condition. This application does not limit recursive state estimation to Kalman filtering; other state estimation methods that can recursively update the time delay state based on the current measurement and historical state can also use corresponding state uncertainty, measurement weight, state update gain, or other equivalent parameters to achieve response enhancement during the end-fire acquisition stage.

[0137] In summary, this application can distinguish between reliable end-fire enhancement measurements and occasional end-fire measurements caused by noise, reverberation, or erroneous correlation peaks by combining the degree of end-fire enhancement of the current time delay measurement relative to the existing smooth time delay state and the reliability of the current measurement. It also increases the influence of the current time delay measurement value on the smooth time delay state value update result only when the corresponding end-fire enhancement conditions are met. This avoids the source azimuth tracking process from being unable to respond promptly to reliable end-fire measurements due to long-term stable wide-side states or the use of conservative end-fire measurement noise settings. It allows the smooth time delay state value to converge more quickly to the time delay measurement value corresponding to the end-fire direction, thereby improving the dual-microphone array's ability to acquire end-fire directions and reducing the risk of state jumps that may result from unconditionally relaxing the end-fire measurement weights.

[0138] In one possible implementation, where the recursive state estimation employs a one-dimensional Kalman filter, this application can use the smoothed delay state value as the state variable to be estimated, and the delay measurement value of the current audio frame as the current measurement value. The prediction and update process of the one-dimensional delay state can be expressed as:

[0139] in, This represents the state uncertainty corresponding to the smoothed delay state value before the update; This represents the process noise corresponding to the state prediction process; This represents the uncertainty of the prediction state corresponding to the current audio frame; This represents the measurement noise corresponding to the delay measurement of the current audio frame. Measurement noise represents the estimation of the error of the current delay measurement by the recursive state estimation process. The greater the measurement noise, the less reliable the current measurement is, and the smaller the state update gain corresponding to the current measurement is usually.

[0140] This indicates the state update gain corresponding to the current audio frame; This indicates the smoothed delay state value before the update; This represents the updated smoothed delay state value; This represents the difference between the current audio frame's latency measurement and the smoothed latency state value before the update. This represents the uncertainty of the state after the state update.

[0141] Thus, during the end-fire acquisition phase, both raising the lower limit of P and lowering R will increase the Kalman gain K, thereby making the smoothing delay approach the reliable end-fire measurement more quickly.

[0142] Step 103: Determine the directional information of the sound source based on the updated smooth delay state value and the microphone spacing of the dual-microphone array.

[0143] In this embodiment, the smooth delay projection is determined based on the updated smooth delay state value, the distance between the two microphones, and the sound speed. The azimuth information of the sound source relative to the dual-microphone array is determined based on the geometric mapping relationship between the smooth delay projection and the sound source azimuth.

[0144] For example, when the end-fire direction on one side of the dual-microphone array is taken as the 0° direction and the end-fire direction on the other side is taken as the 180° direction, the azimuth angle of the sound source can be determined according to the following relationship:

[0145] in, This indicates the azimuth angle of the sound source relative to the dual-microphone array, and the azimuth angle can range from 0° to 180°.

[0146] It is worth noting that in this embodiment, the zero-degree direction of the sound source azimuth and the positive and negative directions of the smooth time delay state value can be preset according to the channel order of the two microphones. In other embodiments, the sound source azimuth can also be determined according to the equivalent inverse cosine mapping, arcsine mapping, lookup table, or pre-calibrated time delay and azimuth mapping relationship. This embodiment does not limit this.

[0147] In one possible implementation, the azimuth information of the sound source in the embodiments of this application includes, but is not limited to, the azimuth angle of the sound source relative to the dual-microphone array, a valid flag representing the effective azimuth of the sound source, and the quality score of the sound source.

[0148] In one possible implementation, embodiments of this application will determine the updated smooth delay projection based on the updated smooth delay state value, and determine the azimuth angle of the sound source relative to the dual-microphone array based on the geometric mapping relationship between the smooth delay projection and the sound source position.

[0149] For example, the updated smoothed delay state value in this application is the delay state value obtained by recursively estimating the delay measurement value of the current audio frame during the sound source azimuth tracking process in normal tracking state, end-fire acquisition state, or end-fire hold state. Compared with directly calculating the sound source azimuth angle based on the delay measurement value of the current audio frame, determining the sound source azimuth angle based on the updated smoothed delay state value can reduce the impact of noise, sound reflection, multipath propagation, or erroneous correlation peaks in a single audio frame on the azimuth output result.

[0150] In one possible implementation, the propagation path difference of the sound signal reaching the two microphones can be determined based on the geometric relationship between the microphone spacing and the azimuth angle of the sound source. Taking the end-fire direction of one side of the dual-microphone array as the zero-degree reference direction of the azimuth angle, the smooth time-delay projection and the azimuth angle of the sound source in this application can satisfy a cosine mapping relationship. Therefore, the azimuth angle of the sound source can be determined by performing an inverse cosine operation on the smooth time-delay projection, as shown in the following formula:

[0151] in, This indicates the azimuth angle of the sound source relative to the dual-microphone array. This represents the updated smoothed time-delay projection. Under this azimuth definition, the source azimuth ranges from 0° to 180°. When the smoothed time-delay projection is close to positive one, the source azimuth is close to 0°, corresponding to the end-fire direction on one side of the dual-microphone array; when the smoothed time-delay projection is close to zero, the source azimuth is close to 90°, corresponding to the wide-side direction of the dual-microphone array; when the smoothed time-delay projection is close to negative one, the source azimuth is close to 180°, corresponding to the end-fire direction on the other side of the dual-microphone array.

[0152] It should be noted that the positive and negative directions of the smooth time delay projection depend on the channel arrangement order of the two microphones and the definition of the positive and negative directions of the time delay. For example, if the time delay measurement is defined as the difference between the arrival time of the sound corresponding to the first microphone and the arrival time of the sound corresponding to the second microphone, then the array sides corresponding to the positive and negative time delays are determined by this definition. Under another channel arrangement order or time delay definition, an equivalent mapping relationship with the opposite direction to the above cosine mapping relationship can also be used to determine the azimuth angle of the sound source. This is as long as a consistent direction definition is used in the time delay measurement, smooth time delay state update, smooth time delay projection calculation, and sound source azimuth angle determination processes.

[0153] In one possible implementation, after determining the azimuth angle of the sound source, this application can further determine that the sound source is in a valid azimuth and generate a valid flag when the measurement confidence level is greater than the confidence threshold, the average signal voltage of the two microphones reaches a preset level threshold, the signal energy ratio of the two microphones meets a preset energy balance condition, and the azimuth angle is within the azimuth range corresponding to the array structure of the dual-microphone array.

[0154] For example, the signal levels of the two microphones can be determined based on the amplitude or energy of the first and second audio frames in the current audio frame, respectively. A preset level threshold can be used to determine whether the audio signals acquired by each channel have sufficient effective signal strength. When the signal level corresponding to either microphone is lower than the preset level threshold, the current audio frame may primarily contain background noise, or the sound source signal strength may be insufficient to support reliable time difference of arrival estimation. When the signal levels of both microphones reach the preset level threshold, further directional validity determinations can be performed.

[0155] For example, the signal energy ratio of the two microphones can be determined based on the signal energy of the first audio frame and the signal energy of the second audio frame. For instance, the ratio of the signal energy of the first audio frame to the signal energy of the second audio frame can be used as the signal energy ratio, or the ratio of the larger signal energy to the smaller signal energy can be used as the signal energy ratio. A preset energy balance condition can be used to limit the allowable energy difference range between the two microphone signals. If the signal energy difference between the two microphones is too large, the current audio frame may have single-channel distortion, abnormal channel gain, microphone obstruction, or other conditions unsuitable for dual-microphone delay estimation. If the signal energy ratio of the two microphones meets the preset energy balance condition, it means that the two audio signals have basic channel consistency for dual-microphone delay estimation and location determination.

[0156] For example, the azimuth range corresponding to the azimuth angle of the sound source can be determined based on the array structure of the dual-microphone array and the preset azimuth reference direction. For instance, the azimuth range corresponding to the array structure can be from 0° to 180°. This application can identify unexplainable azimuth calculation results caused by numerical anomalies, mapping anomalies, or parameter configuration errors by determining whether the azimuth angle of the sound source is within this azimuth range.

[0157] For example, if the azimuth angle of the sound source fails the validity check, this application can retain the calculated azimuth angle of the sound source and mark it as invalid, without outputting the invalid azimuth angle. In this way, subsequent voice interaction modules, beam control modules, or other functional modules that use the sound source azimuth information can determine whether to use the azimuth angle corresponding to the current audio frame based on the validity flag.

[0158] In one possible implementation, this application will determine the quality score of the sound source based on the measurement confidence level of the current audio frame.

[0159] For example, the quality score represents the quality level or reliability of the current sound source azimuth. This application can convert measurement confidence into a quality score based on a preset mapping relationship between confidence and quality score. The mapping relationship may include, but is not limited to, linear mapping, piecewise mapping, and lookup tables, etc., and this application does not specifically limit this.

[0160] In summary, this application can determine the azimuth angle of the sound source within the range of 0°-180° based on the updated smoothed delay state value and the geometric relationship of the dual-microphone array. By measuring the reliability, dual-microphone signal level, dual-microphone signal energy balance, and the physically interpretable range of the azimuth angle, the validity of the determined azimuth angle is judged. This reduces the possibility of misuse of the sound source azimuth result under conditions such as low signal level, dual-microphone channel energy imbalance, insufficient reliability of delay measurement, or abnormal azimuth calculation. Furthermore, this application can output a validity flag and quality score, allowing subsequent functional modules to simultaneously obtain the azimuth result, its validity, and its reliability, thereby improving the completeness and usability of the sound source azimuth information output.

[0161] In one possible implementation, the sound source location determination method provided in this application embodiment can be implemented using an embedded audio processing library. This embedded audio processing library can be written in C language and deployed in a microcontroller, system-on-a-chip, or other electronic device with digital audio processing capabilities, such as an ESP32 series chip. The electronic device can acquire synchronized two-channel Pulse Code Modulation (PCM) audio data from a dual-microphone array and sequentially perform processing such as delay estimation, end-fire state judgment, smooth delay state value update, sound source location calculation, and location output control. This embedded audio processing library can be connected to a pickup link or to an audio processing link combining pickup and amplification. The solution in this application has been implemented in an embedded audio processing library and has been verified in fixed audio playback and synthesis scenarios in the development environment and personal computer simulation environment corresponding to the target chip.

[0162] Reference Appendix Figure 4 The diagram illustrates the mean absolute error (MAE) under different verification scenarios provided by embodiments of this application. In the personal computer synthesis scenario verification, the comparative embodiment and the embodiment of this application use the same dual-microphone audio input, dual-microphone array parameters, and basic sound source azimuth estimation processing. The difference lies in that the embodiment of this application enables the aforementioned end-fire acquisition and end-fire holding mechanisms, while the comparative embodiment does not enable these mechanisms. The verification uses a dual-microphone linear array with a microphone spacing of 65mm, a sampling rate of 16 kHz, fast response parameter presets, a maximum analysis frequency of 4 kHz, and a sound source azimuth scanning step size of 15°. The mean absolute error (MAE) characterizes the average absolute deviation between the sound source azimuth estimation result and the corresponding reference azimuth.

[0163] As attached Figure 4 As shown, the embodiments of this application can maintain sound source azimuth determination accuracy comparable to or better than the comparative embodiments in scenarios such as clean signals, multipath propagation, and reverberation, and significantly reduce the mean absolute error of sound source azimuth determination results in end-fire scenarios and time-varying interference scenarios. Therefore, the embodiments of this application, by identifying the end-fire acquisition state and adjusting the state estimation parameters when entering the end-fire acquisition state, can improve the influence of the current reliable time delay measurement value on the smooth time delay state value update result, making the smooth time delay state value respond more promptly to changes in sound source azimuth, thereby improving the acquisition capability of the end-fire direction and enhancing the adaptability of the sound source azimuth tracking process to time-varying interference.

[0164] It should be noted that the above comparative embodiments are existing implementations in the same development environment without enabling the end-fire acquisition and end-fire hold mechanisms. They are used for comparison with the embodiments of this application and do not imply that the existing implementation belongs to the industry standard algorithm. The above verification results show that the embodiments of this application can achieve targeted improvements in the set end-fire scenarios and time-varying interference scenarios, and also show certain improvements in multipath scenarios and reverberation scenarios; for the set noise interference scenarios and sub-band inconsistency scenarios, the processing parameters or corresponding noise suppression mechanisms can still be further adjusted. Therefore, the above verification results should not be construed as the embodiments of this application necessarily being superior to the comparative embodiments in all acoustic scenarios.

[0165] In one possible implementation, this application embodiment also provides another playback verification condition, which involves playing back the embodiment using a fixed dual-channel end-fire recording. The audio file used for verification is a dual-channel, 16 kHz, 16-bit PCM waveform audio file format (WAV), and the dual-microphone array is an ESP32-S3-Korvo-2 dual-microphone linear array with a microphone spacing of 65 mm. During recording, the distance between the sound source and the dual-microphone array is approximately 30 cm. The first half of the recording is generated by the sound source located at one end of the dual-microphone array, and the second half is generated by the sound source located at the other end of the dual-microphone array. The overall root mean square level difference between the left and right channels of the dual-channel audio file is approximately -0.11 dB, and no clipping is detected.

[0166] With this fixed audio playback configuration, without end-fire acquisition and end-fire hold mechanisms enabled, the effective sound source azimuth output is mainly located in the range of 54.6° to 122.5°. After enabling enhancement processing, including end-fire acquisition and end-fire hold mechanisms, the effective sound source azimuth output can enter the range of approximately 18.1° to 166.4°, and in the two aforementioned speech segments, outputs near end-fires appear at approximately 20° to 35° and approximately 155° to 165°, respectively. Clearly, when the recorded sound sources are located at opposite ends of the dual-microphone array, enabling enhancement processing allows for the output of sound source azimuths that are closer to the corresponding end-fire regions.

[0167] Based on the same inventive concept, and in conjunction with the appendix Figure 5 This application provides a detailed description of a sound source location determination device 50. This device can be used to execute the sound source location determination method provided in the foregoing embodiments. The device can be installed in an electronic device or implemented by a microcontroller, digital signal processor, system-on-a-chip, or other processor with audio data processing capabilities executing a computer program in memory. It can reuse existing dual-microphone PCM input, time delay estimation, location output, and quality gating interfaces, facilitating integration into existing embedded audio libraries. The device includes: The dual-channel audio input module 501 is used to acquire two audio data streams corresponding to the two microphones of the dual-microphone array. These two audio data streams can be synchronously acquired PCM audio data. The dual-channel audio input module 501 can perform corresponding frame division on the two audio data streams to obtain the first channel audio frame and the second channel audio frame corresponding to the current audio frame. The first channel audio frame and the second channel audio frame correspond to the same or mutually corresponding audio acquisition time period, so that the time delay estimation module 502 can determine the time difference of the same sound signal arriving at the two microphones.

[0168] The delay estimation module 502 is connected to the dual-channel audio input module 501 and is used to: estimate the delay based on the first channel audio frame and the second channel audio frame to obtain the delay measurement value of the current audio frame and its measurement reliability. The delay estimation module 502 can use GCC-PHAT, band-limited cross-correlation, phase difference estimation or other dual-microphone delay estimation methods, which are not limited in this embodiment.

[0169] The end-fire state determination module 503 is connected to the delay estimation module 502 and is used to determine the processing state of the sound source orientation tracking process based on the delay measurement value, smooth delay state value, and measurement confidence level of the current audio frame. The processing state can include normal tracking state, end-fire acquisition state, and end-fire hold state. Specifically, when the measurement confidence level meets the preset confidence level requirement, and the sound source orientation tracking process meets at least one of the end-fire enhancement condition and the translateral change condition, the sound source orientation tracking process is determined to have entered the end-fire acquisition state.

[0170] The end-fire state determination module 503 can also normalize the time delay measurement value and the smooth time delay state value according to the ratio of the sound speed to the distance between the two microphones, so as to obtain the time delay measurement projection and the smooth time delay projection. Based on the absolute value change, sign relationship and measurement reliability of the time delay measurement projection and the smooth time delay projection, it can determine whether the sound source orientation tracking process has entered the corresponding processing state.

[0171] The delay tracking module 504 is connected to the end-fire state determination module 503 and is used to perform recursive state estimation on the delay measurement value of the current audio frame according to the processing state determined by the end-fire state determination module 503, so as to obtain the updated smooth delay state value.

[0172] When the sound source orientation tracking process enters the end-fire acquisition state, the time delay tracking module 504 can adjust the state estimation parameters to increase the influence of the time delay measurement value on the smooth time delay state value update result.

[0173] The geometric orientation conversion module 505 is connected to the time delay tracking module 504 and is used to determine the azimuth angle of the sound source relative to the dual-microphone array based on the updated smooth time delay state value, the distance between the two microphones, and the sound speed.

[0174] The output control module 506 is connected to the geometric orientation conversion module 505. It determines the validity of the sound source azimuth angle based on the measurement reliability of the current audio frame, the signal levels of the two microphones, the signal energy ratio of the two microphones, and whether the sound source azimuth angle is within the azimuth range corresponding to the dual-microphone array, and generates a corresponding valid flag. The output control module 506 can also determine the quality score corresponding to the sound source azimuth angle based on the measurement reliability of the current audio frame. The final azimuth information output by the sound source orientation determination device can include the sound source azimuth angle, valid flag, and quality score, for use in voice interaction, sound pickup control, or other subsequent functions.

[0175] It should be noted that the above-described device can execute any of the implementation methods provided in the foregoing embodiments, and will not be described again here. The above module division is only an exemplary functional division method and is not intended to limit the specific hardware structure of the sound source location determination device. In actual implementation, multiple modules can be merged into one processing module, and a module can also be further divided into multiple sub-modules according to the processing function. Each module can be implemented by a processor executing a computer program in memory, or it can be implemented by a digital signal processor, application-specific integrated circuit, field-programmable gate array, or other digital signal processing hardware.

[0176] Please see Figure 6 As shown, based on the same technical concept, this application also provides an electronic device 60. In one embodiment, the electronic device can be a device specifically for determining the location of a sound source. Figure 6 As shown, it includes a memory 601, a communication module 603, and one or more processors 602.

[0177] The memory 601 is used to store computer programs executed by the processor 602. The memory 601 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run various functions; the data storage area may store various information and operation instruction sets.

[0178] Memory 601 may be volatile memory, such as random-access memory (RAM); memory 601 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 601 may be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 601 may be a combination of the above-described memories.

[0179] The processor 602 may include one or more central processing units (CPUs) or digital processing units, etc. The processor 602 is used to implement the aforementioned method for determining the sound source location of the dual-microphone array when it calls the computer program stored in the memory 601.

[0180] The communication module 603 is used to communicate with other terminals and systems.

[0181] This application embodiment does not limit the specific connection medium between the memory 601, the communication module 603, and the processor 602. This application embodiment... Figure 6 The memory 601 and the processor 602 are connected via a bus 604, and the bus 604 is in Figure 6 The connections between other components are shown in bold lines only and are not intended to be limiting. The 604 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 6 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0182] The memory 601 stores a computer storage medium, which stores computer-executable instructions. The computer-executable instructions are used to implement the sound source location determination method of the dual-microphone array according to the embodiments of this application. The processor 602 is used to execute the sound source location determination method of the dual-microphone array according to the above embodiments.

[0183] Based on the same inventive concept, embodiments of this application also provide a storage medium storing a computer program that, when run on a computer, causes the computer to perform the steps in the method for determining the sound source location of a dual-microphone array according to various exemplary embodiments of this application described above.

[0184] In some possible implementations, various aspects of the dual-microphone array sound source location determination method provided in this application can also be implemented in the form of a computer program product, which includes a computer program that, when run on an electronic device, causes the electronic device to perform the steps in the dual-microphone array sound source location determination method according to various exemplary embodiments of this application described above. For example, the electronic device can perform the steps of each embodiment.

[0185] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0186] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this application, the readable storage medium may be any tangible medium that contains or stores a program, and the computer program included therein may be used by or in conjunction with a command execution system, apparatus, or device.

[0187] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0188] Computer programs contained on readable media may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0189] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages.

[0190] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0191] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0192] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0193] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0194] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for determining the location of a sound source using a dual-microphone array, characterized in that, The dual-microphone array includes two microphones spaced apart, and the method includes: Based on the current audio frame's delay measurement value, smooth delay state value, and measurement reliability, determine whether the sound source orientation tracking process has entered the end-fire acquisition state. When the sound source orientation tracking process enters the end-fire acquisition state, the state estimation parameters are adjusted, and the smooth delay state value is updated based on the adjusted state estimation parameters. The state estimation parameters are used to control the degree of influence of the delay measurement value on the update result of the smooth delay state value. Based on the updated smooth delay state value and the microphone spacing of the dual-microphone array, the directional information of the sound source is determined.

2. The method as described in claim 1, characterized in that, The determination of whether the sound source orientation tracking process has entered the end-fire acquisition state includes: When the measurement confidence level meets the preset confidence level requirement, and the sound source orientation tracking process meets at least one of the end-fire enhancement condition and the translateral change condition, the sound source orientation tracking process is determined to enter the end-fire acquisition state.

3. The method as described in claim 2, characterized in that, The end-fire enhancement condition is characterized by the fact that the sound source orientation corresponding to the time delay measurement value is further closer to the end-fire direction than the sound source orientation corresponding to the smooth time delay state value, wherein the end-fire direction is the sound source direction extending along the spacing direction of the two microphones.

4. The method as described in claim 2, characterized in that, The cross-side variation condition is characterized by the fact that the sound source azimuth corresponding to the time delay measurement value and the smooth time delay state value are located on different sides of the dual-microphone array.

5. The method as described in claim 2, characterized in that, Before determining whether the sound source location tracking process has entered the end-fire acquisition state based on the delay measurement value, smooth delay state value, and measurement confidence level of the current audio frame, the method further includes: Based on the ratio of sound speed to the distance between the two microphones, the time delay measurement value and the smooth time delay state value are normalized to obtain the time delay measurement projection and the smooth time delay projection.

6. The method as described in claim 5, characterized in that, The end-fire enhancement conditions include: The difference between the absolute value of the time delay measurement projection and the absolute value of the smooth time delay projection reaches a preset increment value, and the absolute value of the time delay measurement projection is greater than a preset end-shot acquisition threshold.

7. The method as described in claim 5, characterized in that, The translateral variation conditions include: The sign of the time delay measurement projection is opposite to that of the smooth time delay projection, and the absolute value of the time delay measurement projection is greater than a preset cross-side change threshold.

8. The method as described in claim 1, characterized in that, The adjustment of the state estimation parameters includes increasing the influence of the time delay measurement value on the update of the smooth time delay state value; The degree of influence of increasing the time delay measurement value on the smooth time delay state value update includes one or more combinations of the following operations: Increase the state uncertainty corresponding to the smoothed delay state value; Increase the process noise corresponding to the state prediction process of the smoothed delay state value; Reduce the measurement noise corresponding to the time delay measurement value; Increase the state update gain corresponding to the time delay measurement value; Increase the measurement weight corresponding to the time delay measurement value; Reduce the noise amplification factor set for end-fire measurements.

9. The method as described in claim 5, characterized in that, After determining that the sound source orientation tracking process has entered the end-fire acquisition state, the method further includes: When the smooth delay state value meets the preset end-fire locking condition, the sound source azimuth tracking process is determined to enter the end-fire holding state; the end-fire holding state indicates that the smooth delay state value is within the preset delay range corresponding to the end-fire direction.

10. The method as described in claim 9, characterized in that, The end-fire locking conditions include: The absolute value of the smoothed delay projection is greater than a preset end-fire holding threshold; and The state uncertainty of the smooth delay state value is less than the preset locking threshold.

11. The method as described in claim 9, characterized in that, After determining that the sound source azimuth tracking process has entered the end-fire hold state, the method further includes: Determine the information between the time delay measurement value and the smoothed time delay state value before the update, as well as the direction of change corresponding to the information; The innovation is constrained based on the direction of change and the upper limit of the innovation constraint; The smoothed delay state value is updated based on the constrained information.

12. The method as described in claim 11, characterized in that, The new information represents the difference between the delay measurement value and the smooth delay state value before the update, and the direction of change represents the trend of the absolute value of the smooth delay state value when the smooth delay state value is updated based on the new information.

13. The method as described in claim 11, characterized in that, The constraint on the innovation based on the direction of change and the upper limit of the innovation constraint includes: When the change direction is the first change direction, the innovation is constrained based on the upper limit of the first innovation constraint; the first change direction represents the change direction in which the azimuth of the sound source changes towards the end-emitting direction, causing the absolute value of the smooth time delay state value to increase.

14. The method as described in claim 13, characterized in that, The constraint on the innovation based on the change direction and the upper limit of the innovation constraint further includes: When the change direction is the second change direction, the innovation is constrained based on the upper limit of the second innovation constraint; the second change direction represents the change direction in which the sound source orientation changes towards the wide side direction, causing the absolute value of the smooth time delay state value to decrease, the wide side direction is the sound source direction perpendicular to the end-fire direction, and the upper limit of the first innovation constraint is greater than the upper limit of the second innovation constraint.

15. The method as described in claim 11, characterized in that, Before constraining the innovation based on the direction of change and a preset upper limit for innovation constraints, the method further includes: Based on the smooth time delay projection, determine the geometric mapping sensitivity corresponding to the smooth time delay state value; Based on the preset angle change amount and the geometric mapping sensitivity, the delay change budget of the current audio frame is determined; Based on the time delay variation budget, the first direction coefficient, and the second direction coefficient, the first and second innovation constraint upper limits are determined respectively; the first direction coefficient is greater than the second direction coefficient.

16. The method as described in claim 11, characterized in that, When the sound source azimuth tracking process is in the end-fire holding state and the cross-side change condition is met, the innovation constraint corresponding to the end-fire holding state is not executed, or the upper limit of the innovation constraint is increased. The smooth delay state value is updated based on the state estimation parameters corresponding to the end-fire acquisition state.

17. The method as described in claim 1, characterized in that, Before determining whether the sound source location tracking process has entered the end-fire acquisition state based on the delay measurement value, smooth delay state value, and measurement confidence level of the current audio frame, the method further includes: The dual-microphone audio data of the dual-microphone array is synchronously framed to obtain the first channel audio frame and the second channel audio frame corresponding to the current audio frame; Delay estimation is performed on the first channel audio frame and the second channel audio frame to obtain the delay measurement value and the measurement confidence level; Obtain the delay measurement value corresponding to the previous audio frame of the current audio frame, and perform recursive state estimation on the delay measurement value of the previous audio frame to obtain the smooth delay state value.

18. The method as described in claim 1, characterized in that, Before determining whether the sound source location tracking process has entered the end-fire acquisition state based on the delay measurement value, smooth delay state value, and measurement confidence level of the current audio frame, the method further includes: Based on the speed of sound and the distance between the two microphones, the physical delay range corresponding to the dual-microphone array is determined; When the delay measurement value exceeds the physical delay range, the delay measurement value is limited to the boundary value corresponding to the physical delay range.

19. The method as described in claim 1, characterized in that, The determination of the sound source's directional information based on the updated smooth delay state value and the microphone spacing of the dual-microphone array includes: Based on the updated smooth delay state value, determine the updated smooth delay projection; Based on the geometric mapping relationship between the smooth time delay projection and the sound source location, the azimuth angle of the sound source relative to the dual-microphone array is determined.

20. The method as described in claim 19, characterized in that, The step of determining the directional information of the sound source based on the updated smooth delay state value and the microphone spacing of the dual-microphone array further includes: When the measurement confidence level is greater than the confidence level threshold, the signal voltages of the two microphones reach a preset level threshold, the signal energy ratio of the two microphones meets a preset energy balance condition, and the azimuth angle is within the azimuth range corresponding to the array structure of the dual-microphone array, the sound source is determined to be in a valid azimuth, and a valid flag is generated.

21. The method as described in claim 19, characterized in that, The step of determining the directional information of the sound source based on the updated smooth delay state value and the microphone spacing of the dual-microphone array further includes: The quality score of the sound source is determined based on the measurement confidence level of the current audio frame.

22. An electronic device, characterized in that, include: At least one processor; as well as A memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, perform the method according to any one of claims 1 to 21.

23. A non-transitory computer-readable storage medium, characterized in that, The device stores computer-executable instructions that, when executed by at least one processor of the device, cause the device to perform the method according to any one of claims 1 to 21.

24. A computer program product, characterized in that, It includes computer-executable instructions that, when executed by at least one processor of the device, cause the device to perform the method according to any one of claims 1 to 21.

Images