Pickup method, portable device, computer storage medium, and program product

Abstract

This application provides a sound pickup method, a portable device, a computer storage medium, and a program product. The sound pickup method includes: acquiring ambient sound signals using a triangular microphone array, including a first microphone pair distributed along an axial direction and a second microphone pair distributed along a lateral direction, the axial direction and the lateral direction being perpendicular to each other; performing differential processing on the sound signals acquired by the axial microphones and the sound signals acquired by the lateral microphones respectively to obtain differential signals; and performing adaptive filtering based on the differential signals to obtain a target sound signal. This application effectively solves the problem of portable devices struggling to clearly pick up the voice of a target speaker in complex acoustic environments.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to a sound pickup method, a portable device, a computer storage medium, and a computer program product. Background Technology

[0002] With the widespread adoption of intelligent technologies, there is an increasing demand for devices that can conveniently capture sound in real time and perform subsequent intelligent processing based on the captured sound data. As a result, portable devices such as AI pens, which can capture sound in real time and translate or generate intelligent summaries, are being used more and more frequently.

[0003] However, real-world sound environments are often complex, especially in open or noisy spaces. Noise and interference make it difficult for portable devices to clearly pick up the voice of the target speaker, while traditional hardware devices that use multi-channel directional sound pickup are difficult to miniaturize and deploy. Therefore, how to enable portable devices to effectively pick up the voice of the target speaker has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of this application provide a sound pickup scheme to at least partially solve the above-mentioned problems.

[0005] According to a first aspect of the embodiments of this application, a sound pickup method is provided, comprising: acquiring sound signals in the environment through a triangular microphone array, the triangular microphone array including a first pair of microphones distributed along an axial direction and a second pair of microphones distributed along a transverse direction, the axial direction and the transverse direction being perpendicular to each other; performing differential processing on the sound signals acquired by the microphones in the axial direction and the sound signals acquired by the microphones in the transverse direction respectively to obtain differential signals; and performing adaptive filtering based on the differential signals to obtain a target sound signal.

[0006] According to a second aspect of the present application, a portable device is provided, comprising at least: an audio processing chip, a triangular microphone array, and a memory; the audio processing chip is communicatively connected to the triangular microphone array and the memory; the triangular microphone array is used to collect sound signals from the environment, the triangular microphone array including a first pair of microphones distributed along an axial direction and a second pair of microphones distributed along a lateral direction, the axial direction and the lateral direction being perpendicular to each other; the memory is used to store at least one executable instruction, the executable instruction causing the audio processing chip to perform an operation corresponding to the sound pickup method described in the first aspect based on the sound signals collected by the triangular microphone array.

[0007] According to a third aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0008] According to a fourth aspect of the embodiments of this application, a computer program product is provided, including computer instructions that instruct a computing device to perform an operation corresponding to the method described in the first aspect.

[0009] According to the solution provided in this application embodiment, taking into account the miniaturization characteristics of portable devices, a triangular microphone array is used, and the spatial position relationship of the three microphones forms mutually perpendicular axial and lateral directions. Then, differential processing is performed on the sound signals collected by the microphones in the two directions respectively, generating an axial differential signal sensitive to the axial direction and a lateral differential signal sensitive to the lateral direction, thereby achieving preliminary separation of the spatial components of the sound field at the signal level. Based on this, the lateral differential signal, which serves as an interference reference, is eliminated from the axial differential signal through adaptive filtering, thereby obtaining the target sound signal in the axial direction. Thus, on the one hand, effective directional sound pickup can be achieved with only three microphones, with lower requirements for device size and power consumption, making it suitable for deployment in miniaturized portable devices such as recording cards and AI glasses; on the other hand, the differential processing and adaptive filtering of the sound signals can be implemented through algorithms, eliminating the need for additional channels or directional sound pickup hardware, further reducing the requirements for device size and power consumption. Based on this, the solution in this application embodiment effectively solves the problem of portable devices struggling to clearly pick up the voice of a target speaker in complex acoustic environments. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0011] Figure 1A This is a flowchart illustrating the steps of a sound pickup method according to an embodiment of this application; Figure 1B for Figure 1A A schematic diagram of the acoustic deployment of a triangular microphone array in the illustrated embodiment; Figure 1C for Figure 1A A schematic diagram of a frequency band processing strategy in the illustrated embodiment; Figure 1D for Figure 1A A schematic diagram illustrating an example of a sound pickup scenario in the illustrated embodiment; Figure 2 This is a schematic diagram of the structure of a portable device according to an embodiment of this application. Detailed Implementation

[0012] To enable those skilled in the art to better understand the technical solutions in the embodiments of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this application.

[0013] The specific implementation of the embodiments of this application will be further described below with reference to the accompanying drawings.

[0014] Reference Figure 1A The diagram illustrates a flowchart of a sound pickup method according to an embodiment of this application. The sound pickup method of this embodiment includes the following steps: Step S102: Collect sound signals from the environment using a triangular microphone array.

[0015] The triangular microphone array is an array of three microphones, including a first pair of microphones distributed along the axial direction and a second pair of microphones distributed along the lateral direction, wherein the axial and lateral directions are perpendicular to each other. In this embodiment, the three microphones can be omnidirectional microphones.

[0016] For ease of explanation, the following will be combined with Figure 1B The feasible acoustic deployment of a triangular microphone array is explained.

[0017] Figure 1B In this design, three omnidirectional microphones are used to form a triangular microphone array. Omnidirectional microphones have almost identical sensitivity to sound from all directions, and their pickup pattern is spherical (3D) or circular (2D polar coordinates). The uniform omnidirectional characteristics of the omnidirectional microphones themselves ensure the accuracy and controllability of the directionality constructed in subsequent differential processing, avoiding mutual interference between the microphone's own directional characteristics and the differential directionality. The three omnidirectional microphones... Figure 1B The three mics are denoted as mic1, mic2, and mic3, and are arranged in a triangular topology within portable devices (including but not limited to smart cards, portable recording cards, AI glasses, smartwatches, smart earphones, AI voice recorders, and other miniaturized portable devices).

[0018] exist Figure 1BIn the first and second diagrams, mic1 and mic2 (the first microphone pair) are distributed along the axial direction of the portable device. mic1 can be located at the front of the portable device, and mic2 can be located at the rear or center. The line connecting them forms the axial direction of the portable device, i.e., the main propagation direction of the target sound. mic2 and mic3 (the second microphone pair) are distributed along the lateral direction of the portable device. The line connecting them is perpendicular to the axial direction, forming the lateral direction of the portable device, i.e., the main source direction of the interfering sound. The distance between the two microphones in the axial direction is denoted as the first distance, and the distance between the two microphones in the lateral direction is denoted as the second distance. In one feasible approach, the range of the first distance and the range of the second distance are both 1-10cm, but the first distance and the second distance are not equal. Figure 1B In the first illustration, the first distance is greater than the second distance; in the second illustration, the second distance is greater than the first distance. However, this is only an example. In practical applications, the first distance in the first illustration may be less than the second distance, and the second distance in the second illustration may be less than the first distance. By using unequal first and second distances, the microphone can form differentiated spatial sampling rates in different directions. For example, a larger first distance in the axial direction can enhance the phase difference detection capability of low-frequency signals (due to the longer wavelength), while a smaller second distance in the lateral direction can improve the resolution of high-frequency signals. This non-uniform sampling can cover a wider frequency band and reduce the ambiguity of direction estimation. Furthermore, compared to uniform arrays (such as equilateral triangles) that may generate grating lobes (false peaks) at specific frequencies, leading to incorrect direction estimation, this unequal spacing design of the embodiments of this application can destroy periodicity and suppress grating lobe generation, especially in the high-frequency band, effectively improving the robustness of angle estimation. In addition, since a larger spacing is more suitable for low-frequency signals, while a smaller spacing is more suitable for high-frequency signals, this unequal spacing design can also effectively suppress aliasing over a wide frequency band. However, it should be noted that the solution of this application embodiment can also be applied to symmetrical triangular microphone arrays with equal first and second distances, or can be applied to symmetrical triangular microphone arrays after simple modifications.

[0019] exist Figure 1BIn the third and fourth diagrams, mic1 and mic2 (the first microphone pair) are distributed along the axial direction of the portable device. mic1 can be located at the front of the portable device, and mic2 can be located at the rear or center. The line connecting them forms the axial direction of the portable device, i.e., the main propagation direction of the target sound. mic1 and mic3 (the second microphone pair) are distributed along the lateral direction of the portable device. The line connecting them is perpendicular to the axial direction, forming the lateral direction of the portable device, i.e., the main source direction of the interfering sound. The first distance between the two microphones in the axial direction and the second distance between the two microphones in the lateral direction both range from 1-10cm, and the first and second distances are not equal. Figure 1B In the third illustration, the first distance is greater than the second distance; in the fourth illustration, the second distance is greater than the first distance. However, this is only an example. In practical applications, the first distance in the third illustration may be less than the second distance, and the second distance in the fourth illustration may be less than the first distance.

[0020] Through the aforementioned triangular topology deployment, the three omnidirectional microphones form a pickup array with a clear spatial directionality. In actual deployment, those skilled in the art can flexibly determine the installation positions and specific spacing in each direction of the three omnidirectional microphones according to the specific shape and size constraints of the portable device; this application embodiment does not impose such limitations.

[0021] In real-world sound pickup scenarios, the acoustic environment is often a complex mix of multiple sound sources. Taking a face-to-face communication scenario as an example, the voice emitted by the speaker directly in front of the portable device is the target sound signal to be picked up. However, ambient noise from the side of the portable device, the voices of irrelevant people, and sounds from other directions of the portable device, such as the rear, all constitute interference signals that need to be suppressed. To address this, in the triangular microphone array of this application embodiment, three microphones are deployed with mutually orthogonal spatial pickup directions, namely the axial direction and the lateral direction. The axial direction corresponds to the longitudinal axis of the portable device, such as the straight line connecting the front and rear of the portable device. As mentioned earlier, this is usually the main propagation direction of the target sound signal (such as the voice signal of the other party in a face-to-face communication scenario). The lateral direction is perpendicular to the axial direction and corresponds to the transverse axis of the portable device, i.e., the direction connecting the two sides of the portable device. This is also the main source direction of ambient noise and the voice signals of irrelevant people. By deploying three microphones in these two orthogonal directions, the triangular microphone array can simultaneously sense sound signals from the axial and lateral directions, thereby achieving synchronous capture of the spatial components of the sound field in two orthogonal dimensions with the fewest microphones, and providing a basis for subsequent differential processing in the two directions.

[0022] Step S104: Perform differential processing on the sound signals collected by the microphone in the axial direction and the sound signals collected by the microphone in the lateral direction to obtain differential signals.

[0023] For the aforementioned triangular microphone array, the three sound signals collected by its three omnidirectional microphones do not possess spatial directional selectivity; that is, each signal simultaneously mixes sound components from various directions. To physically decompose the different spatial components of the sound field and obtain directional sound signals, this embodiment employs a differential processing method.

[0024] In differential processing, for any two microphones in a triangular microphone array, when the sound source is located in the extended direction of the line connecting the two microphones (i.e., end-fire direction), the sound waves reach the two microphones with the greatest propagation time difference, corresponding to the greatest sound pressure difference, and the signal amplitude after differential processing is the largest. However, when the sound source is located perpendicular to the line connecting the two microphones (i.e., side-fire direction), the sound waves reach the two microphones almost simultaneously, the sound pressure difference between them approaches zero, and the signal amplitude after differential processing approaches zero, effectively suppressing the sound components in that direction. This differential processing method exhibits bidirectional end-fire directivity in space, meaning it is sensitive to signals along the direction of the array connection (both ends) but insensitive to signals perpendicular to the connection.

[0025] Specifically, in the embodiments of this application, for two microphones in the axial direction (such as...) Figure 1B Differential processing of the sound signals collected by mic1 and mic2 (as shown) yields an axial differential signal that is sensitive to the axial direction (front and rear) but insensitive to the lateral direction (side); for the two microphones in the lateral direction (such as...) Figure 1B The sound signals collected by mic2 and mic3 are differentially processed to obtain a lateral differential signal that is sensitive to the lateral direction but insensitive to the axial direction. Based on this, through two sets of orthogonal differential operations, the original three omnidirectional signals are decomposed into two differential signals with mutually orthogonal directivity, realizing the initial separation of the sound field spatial components in the axial and lateral directions.

[0026] Taking axial differential as an example, if there are two microphones in the axial direction, such as Figure 1B The sound signals collected by mic1 and mic2 are denoted as x1 and x2 respectively. After differential processing, the resulting differential signal y1 is: y1 = x1 - x2. Similarly, two microphones in the horizontal direction (such as...) Figure 1B The sound signals collected by mic2 and mic3 shown in the diagram can be subtracted to obtain the difference signal in the horizontal direction, exemplarily represented as y2 = x3 - x2. Here, x3 represents... Figure 1B The sound signal collected by mic3 in the device.

[0027] Step S106: Perform adaptive filtering based on the differential signal to obtain the target sound signal.

[0028] After obtaining two differential signals, namely the axial differential signal and the lateral differential signal, subsequent processing can be performed based on these two differential signals. In this embodiment, adaptive filtering processing is used to obtain the target sound signal.

[0029] However, since the effectiveness of differential processing is related to the signal frequency, differential processing in different frequency ranges will produce different effects. Therefore, in one alternative approach, the frequency band of the audio signal is divided into a first frequency band, a second frequency band, and a third frequency band, wherein the first frequency band is lower than the second frequency band, and the second frequency band is lower than the third frequency band. For example, the first frequency band may correspond to low frequencies, such as below 300Hz; the second frequency band may correspond to mid frequencies, such as 300-1700Hz / 3400Hz (related to the spacing between the two microphones); and the third frequency band may correspond to high frequencies, such as above 1700Hz / 3400Hz. However, this is not the only possibility. In practical applications, those skilled in the art can also divide the frequency bands according to the actual situation, and may divide them into more frequency bands for differentiated processing.

[0030] Optionally, based on this, adaptive filtering based on the differential signal to obtain the target sound signal may include: obtaining the frequency band information of the frequency band where the sound signal is located; if the frequency band information indicates that the frequency band where the sound signal is located is the second frequency band, then differential processing is performed on the differential signal in the axial direction and the differential signal in the lateral direction, and adaptive filtering is performed based on the differential processing result to obtain the target sound signal.

[0031] For the second frequency band, on the one hand, the sound wave wavelength is not negligible relative to the microphone spacing, and on the other hand, the microphone spacing is much smaller than the sound wave wavelength, so spatial aliasing will not occur. Therefore, differential processing in the second frequency band can achieve a sufficiently high signal-to-noise ratio while maintaining accurate spatial directivity. Thus, if the frequency band of the sound signal is the second frequency band, differential processing can be performed on the differential signals in the axial direction and the lateral direction to obtain a more directional sound signal. For example, this differential processing can be first-order differential processing to simplify the implementation, improve sound pickup efficiency, and better adapt to portable devices.

[0032] In the first frequency band, the sound wave wavelength is much larger than the microphone spacing, and the sound pressure signals received by two adjacent microphones are almost the same, resulting in a small sound pressure difference. Therefore, the signal-to-noise ratio of the signal obtained after differential processing may be low, and some effective speech components may be submerged by quantization noise and system background noise. To address this, in one optional embodiment of this application, if the sound signal frequency band is the first frequency band, adaptive filtering is directly performed based on the differential signal corresponding to the larger of the first and second distances to obtain the target sound signal. A larger microphone spacing corresponds to a lower low-frequency cutoff frequency, meaning better coverage in the low-frequency band, retaining more effective signal components in the low-frequency band, and obtaining better low-frequency signal quality.

[0033] For the third frequency band, the sound wave wavelength is comparable to or even smaller than the microphone spacing, which may cause spatial aliasing in the differential array, resulting in signals from different directions being mixed in the differential results. Therefore, in one optional embodiment of this application, if the sound signal's frequency band is the third frequency band, the differential signals in the axial and lateral directions are summed and averaged. Then, adaptive filtering is performed on the averaged sound signal to obtain the target sound signal. By using the summation and averaging method, the limitations of differential processing in the high-frequency band can be avoided, ensuring that the high-frequency signal is in a neutral state in terms of spatial directivity, providing a basis for further directional filtering by the differential adaptive filter.

[0034] Figure 1C An example is shown where the corresponding processing methods described above are applied to the first, second, and third frequency bands, respectively.

[0035] Figure 1C In this context, the second frequency band's range is related to the distance between the two microphones, with its upper limit expressed as f_max = c / (2d), where c ≈ 340 m / s, representing the speed of sound; and d represents the microphone distance in the corresponding direction. For example, if d = 5cm, then f_max = 3400Hz; if d = 10cm, f_max = 1700Hz. Based on this, the frequency band... Figure 1C The frequency band is divided into three bands: the first band is f < 300 Hz, the second band is 300 Hz ≤ f ≤ f_max, and the third band is f > f_max.

[0036] Assume the differential signals in the axial and lateral directions are represented as y1(f) and y2(f), respectively, and the first distance in the axial direction is greater than the second distance in the lateral direction. Then, in the first frequency band, the differential signal in the axial direction corresponding to the larger distance (the first distance) is retained, resulting in y(f) = y1(f). In the second frequency band, the differential signals y1(f) in the axial direction and y2(f) in the lateral direction are further processed by first-order differential processing, resulting in y(f) = y1(f) - y2(f). In the third frequency band, the differential signals y1(f) in the axial direction and y2(f) in the lateral direction are first summed, then averaged, resulting in y(f) = (y1(f) + y2(f)) / 2. Here, f represents the frequency.

[0037] First-order differential processing may introduce phase shift, while summing and averaging will not introduce additional phase shift. Based on this, in one alternative approach, when the frequency band of the sound signal transitions from the second frequency band to the third frequency band, the differential signals in the axial direction and the differential signals in the lateral direction can be summed and averaged. Furthermore, the sound signal obtained after averaging is phase-aligned with the differential signal in the second frequency band. Based on the phase-aligned sound signal, adaptive filtering is performed to obtain the target sound signal, so as to avoid the comb filtering effect when performing weight fusion near f_max when transitioning from the second frequency band to the third frequency band, which would cause a dip in the frequency response of the transition band.

[0038] For example, a complex weight matrix containing amplitude and phase compensation can be constructed first. In the transition frequency band (e.g., f_max±Δf), for frequency f, crossfade-in and fade-out weights w(f) are defined. Before performing fusion near f_max, phase compensation is performed on y_avg(f) (to distinguish it from other signals, the signal y(f) in y(f) = (y1(f)+y2(f)) / 2 is represented as y_avg(f)). For example, y_avg(f) can be input into a first-order all-pass filter or phase compensation network for phase compensation, so that its phase response near f_max approximates the phase response of the y_diff(f) signal (to distinguish it from other signals, the signal y(f) in y(f) = y1(f) - y2(f) is represented as y_diff(f), i.e., the result of differential processing in the second frequency band). The compensated signal is represented as y_avg_comp(f). Based on this, complex domain soft fusion can be performed, which can be expressed as: y_out(f) = (1 - w(f)) * y_diff(f) + w(f) * y_avg_comp(f). This eliminates the comb filtering effect in the transition band.

[0039] After the above processing, the corresponding signal to be adaptively filtered can be obtained, such as the y(f) signal corresponding to the first frequency band (e.g. Figure 1C The signals obtained are: y(f) = y1(f) for the first frequency band, y_diff(f) for the second frequency band, y_out(f) for the transition from the second to the third frequency band, and y_avg(f) for the third frequency band. Then, an adaptive filtering algorithm can be used to perform adaptive filtering on the obtained signals.

[0040] Adaptive filtering algorithms are a class of signal processing algorithms that can dynamically adjust their filtering coefficients based on the statistical characteristics of the input signal. Compared to fixed-coefficient filtering algorithms, adaptive filtering algorithms do not require prior knowledge of the statistical characteristics of the interference signal. Instead, they continuously update the filtering coefficients through a real-time error feedback mechanism, minimizing the mean square error between the output of the filtering algorithm and the target signal. In the embodiments of this application, the adaptive filtering algorithm uses the lateral differential signal as the reference input (interference reference) and the axial differential signal as the main input (a signal containing a mixture of the target sound signal and the interference signal). By continuously estimating and updating the filtering coefficients, the algorithm subtracts the reference input weighted by the filtering coefficients from the main input to obtain a signal that retains only the sound component in the target direction, i.e., the target sound signal.

[0041] In one alternative approach, obtaining the target sound signal by adaptive filtering based on the differential signal may include: obtaining differential signals in the axial direction and differential signals in the lateral direction; calculating the spatial energy ratio of the differential signals in the axial direction and the differential signals in the lateral direction; determining the tracking parameters of the adaptive filtering algorithm based on the spatial energy ratio; and adaptively filtering the differential signal in the axial direction according to the adaptive filtering algorithm and its corresponding tracking parameters to obtain the target sound signal.

[0042] The spatial energy ratio reflects the relative relationship between the sound energy in the lateral direction and the sound energy in the axial direction at the current moment, and can characterize the spatial directionality of the current sound field. In a specific implementation, short-time energy estimates can be performed on the differential signals in the axial direction and the differential signals in the lateral direction (e.g., calculating the mean square value within a certain window length), and then the ratio of the two can be taken to obtain the spatial energy ratio at the current moment. However, it is not limited to this. In practical applications, those skilled in the art can flexibly choose the specific calculation method for the spatial energy ratio, and the embodiments of this application do not impose any restrictions on this.

[0043] In different acoustic scenarios, the intensity and rate of change of interference signals may vary. For example, when the sound from the direction of interference is strong (lateral energy dominates), the adaptive filtering algorithm needs to respond to the rapid changes in the interference signal with a faster tracking speed and update the filter coefficients in a timely manner to suppress sudden interference. When the sound from the direction of the target is strong (axial energy dominates), it indicates that the current acoustic environment is relatively stable and the interference is weak. In this case, the adaptive filtering algorithm should maintain the stability of the filter coefficients with a slower tracking speed to avoid excessive cancellation and distortion of the target sound signal due to excessively fast tracking speed. By coupling the spatial energy ratio with the dynamic adjustment of the tracking parameters of the adaptive filtering algorithm, the timely perception and response of the adaptive filtering algorithm to different acoustic scenarios is realized, thus forming a dynamic parameter control mechanism for the sound field perception-driven adaptive filtering algorithm.

[0044] Considering the performance of portable devices, the adaptive filtering algorithm used in this application embodiment can be a recursive least squares filtering (RLS) algorithm or a Kalman filter algorithm to achieve fast sound signal tracking under limited hardware conditions.

[0045] When using the RLS algorithm, the tracking parameters of the adaptive filtering algorithm can be determined based on the spatial energy ratio as follows: if the spatial energy ratio indicates that the differential signal in the lateral direction is dominant, then the forgetting factor in the RLS algorithm is reduced; if the spatial energy ratio indicates that the differential signal in the axial direction is dominant, then the forgetting factor in the RLS algorithm is increased.

[0046] The Regressive Sequencing Scale (RLS) algorithm utilizes all historical observation data to minimize the sum of squared weighted errors, thereby obtaining the optimal filter coefficient estimate for the current moment. Compared to stochastic gradient descent algorithms (such as LMS), the RLS algorithm converges faster, has stronger tracking ability for non-stationary signals, and is more suitable for sound pickup scenarios where interference signals may change rapidly. In the RLS algorithm, the key tracking parameter is the forgetting factor, usually denoted as λ, with a value range of (0, 1). The forgetting factor represents the degree to which the algorithm weights historical data decay: the smaller the forgetting factor, the faster the weights of historical data decay over time, and the faster the algorithm responds to changes in the current signal (faster tracking speed), but the lower the statistical stability of the filter coefficients; the larger the forgetting factor, the slower the weights of historical data decay, and the algorithm uses data from a longer historical interval to estimate the filter coefficients (slower tracking speed), but the estimation results are more stable and have smaller variance.

[0047] Specifically, in the embodiments of this application, if the spatial energy ratio indicates that the differential signal in the lateral direction is dominant, the forgetting factor in the RLS algorithm is reduced to accelerate the algorithm's tracking speed, enabling the filter coefficients to quickly respond to changes in the interference signal and achieve rapid suppression of sudden interference; if the spatial energy ratio indicates that the differential signal in the axial direction is dominant, the forgetting factor in the RLS algorithm is increased to maintain the stability of the filter coefficients, stably extracting the sound signal in the target direction when the interference is weak, and avoiding false cancellation of the target signal due to excessively fast tracking speed.

[0048] By dynamically adjusting the forgetting factor based on the spatial energy ratio, the RLS algorithm can quickly track and suppress interference when the interference sound changes drastically (by reducing λ to improve response speed), and maintain stable filter coefficients when the acoustic scene is stable (by increasing λ to reduce variance). This effectively suppresses distortion of the axial sound signal while accelerating the convergence of interference noise, achieving a dynamic balance between tracking speed and signal fidelity. In practical applications, those skilled in the art can flexibly design the mapping relationship between the forgetting factor and the spatial energy ratio (such as linear mapping, piecewise linear mapping, or nonlinear mapping, etc.) according to specific scenario requirements; this application does not impose any limitations on this.

[0049] When using the Kalman filter algorithm, the tracking parameters of the adaptive filtering algorithm can be determined based on the spatial energy ratio as follows: if the spatial energy ratio indicates that the differential signal in the lateral direction is dominant, then the process noise covariance in the Kalman filter algorithm is increased; if the spatial energy ratio indicates that the differential signal in the axial direction is dominant, then the process noise covariance in the Kalman filter algorithm is decreased.

[0050] The Kalman filter algorithm is a recursive optimal estimation algorithm based on a state-space model. It achieves continuous optimization of the filter coefficients by dynamically predicting the system state (i.e., the adaptive filter coefficients) and correcting them based on observations. The key tracking parameter of the Kalman filter algorithm is the process noise covariance, usually denoted as Q. The process noise covariance reflects the Kalman filter algorithm's prior assumptions about the degree of change of the filter coefficients over time: the larger the process noise covariance, the more drastic the changes in the filter coefficients are assumed by the Kalman filter algorithm, resulting in lower confidence in the estimated state at the previous time step and a greater reliance on current observation data to update the state estimate, thus accelerating the tracking speed; the smaller the process noise covariance, the more gradual the changes in the filter coefficients are assumed, resulting in higher confidence in historical state estimates, slower tracking speed, but more stable estimation results.

[0051] Specifically, in the embodiments of this application, if the spatial energy ratio indicates that the differential signal in the lateral direction is dominant, the process noise covariance in the Kalman filter algorithm is increased to accelerate the tracking response of the Kalman filter algorithm to changes in the interference signal; if the spatial energy ratio indicates that the differential signal in the axial direction is dominant, the process noise covariance in the Kalman filter algorithm is decreased to maintain the stability of the filter coefficients and stably extract the sound signal in the target direction. Thus, adaptive filtering is achieved, which enables faster tracking when interference is strong and more stable when interference is weak, improving the adaptive filtering algorithm's perception and response capability to changes in the acoustic scene.

[0052] However, this is not the only one. Other adaptive filtering algorithms that can perform similar dynamic adjustment of tracking parameters can also be applied to the embodiments of this application, and the embodiments of this application do not limit this.

[0053] Through the above process, using the differential signal in the lateral direction as the interference reference, the adaptive filtering algorithm eliminates the residual interference components from the differential signal in the axial direction. Through the dynamic adjustment mechanism of the tracking parameters based on the spatial energy ratio, the filtering system can flexibly respond to different acoustic scenarios and output a clean sound pickup result that retains only the target sound signal in the axial direction.

[0054] The following, combined with Figure 1D The above sound pickup process will be explained using a scenario example.

[0055] Figure 1D In this example, a two-person dialogue scenario between a salesperson and a customer is used. However, those skilled in the art should understand that the solution in this application embodiment is also applicable to other dialogue scenarios, such as face-to-face meeting scenarios, multi-person meeting scenarios, and other two-person conversation scenarios.

[0056] Furthermore, in this example, the portable device is implemented as a smart card equipped with the aforementioned triangular microphone array. This smart card can be manually triggered, such as through gesture operations (e.g., double-clicking), voice commands, or physical switches; it can also be automatically detected for activation, such as through sound source localization or voice-activated VAD. This application embodiment does not limit the triggering method of the smart card.

[0057] Once the smart card is triggered, a triangular microphone array collects sound signals from the current environment. These signals include not only the audio of the conversation between the salesperson and the customer, but also ambient noise and the voices of unrelated speakers. Furthermore, the voices emitted by both the salesperson and the customer are broadband signals covering the first, second, and third frequency bands. A frequency band processing strategy is applied to the voice signals of the salesperson and the customer according to their respective frequency bands, without altering the frequency band division based on the speaker's identity.

[0058] After acquiring sound signals, the smart card determines the main speaker for the current time period through endpoint detection and voiceprint feature matching. Simultaneously, it divides the acquired sound signals into frequency bands (e.g., first, second, and third bands) and employs different processing strategies for each band. Specifically, for the second band (the area where human voice energy is concentrated): the smart card performs further differential processing on the axial and lateral differential signals to enhance directivity and highlight the voice of the target person directly in front (such as a salesperson or customer). For the third band (the area where details and noise are mixed): the smart card sums and averages the axial and lateral differential signals to avoid high-frequency spatial aliasing and preserve speech clarity. For the first band (the area dominated by environmental noise): the smart card processes the differential signals from microphones with a larger spacing to suppress low-frequency environmental noise.

[0059] Assuming that from 0:10 to 0:20, the salesperson is speaking, if the smart card detects that axial energy is dominant (e.g., the salesperson is directly in front), it dynamically adjusts the adaptive filtering parameters based on the spatial energy ratio. Then, it performs the aforementioned differential processing on each frequency band of the salesperson's audio signal, synthesizing a clear salesperson's voice. From 0:25 to 0:40, the customer is speaking. If the smart card detects that lateral energy is dominant (i.e., the customer is to the side), it dynamically adjusts the adaptive filtering parameters based on changes in the lateral energy ratio to switch the sound pickup focus. Then, it performs the aforementioned differential processing on each frequency band of the customer's audio signal, synthesizing a clear customer's voice.

[0060] It should be noted that the above is only an illustrative example. In practical applications, the speaker's voice signal may involve some or all of the first, second, and third frequency bands. In such cases, the corresponding processing strategies for different frequency bands should be applied.

[0061] Subsequently, the target audio signal can be processed as needed, including but not limited to: speech-to-text, real-time translation, or intelligent summarization, etc.

[0062] As can be seen, the solution of this application, taking into account the miniaturization characteristics of portable devices, uses a triangular microphone array, and forms mutually perpendicular axial and lateral directions based on the spatial positional relationship of the three microphones. Then, differential processing is performed on the sound signals collected by the microphones in the two directions respectively, generating axial differential signals sensitive to the axial direction and lateral differential signals sensitive to the lateral direction, thereby achieving preliminary separation of the spatial components of the sound field at the signal level. Based on this, the lateral differential signal, which serves as an interference reference, is eliminated from the axial differential signal through adaptive filtering, thus obtaining the target sound signal in the axial direction. Therefore, on the one hand, effective directional sound pickup can be achieved with only three microphones, with low requirements for device size and power consumption, making it suitable for deployment in miniaturized portable devices such as recording cards and AI glasses; on the other hand, the differential processing and adaptive filtering of the sound signals can be implemented through algorithms, without the need for additional channels or directional sound pickup hardware, further reducing the requirements for device size and power consumption. Based on this, the solution of this application effectively solves the problem of portable devices struggling to clearly pick up the voice of the target speaker in complex acoustic environments.

[0063] Reference Figure 2 This application also provides a portable device. The portable device includes at least: an audio processing chip 202, a triangular microphone array 204, and a memory 206.

[0064] The triangular microphone array 204 includes a microphone array deployed in a portable device in a triangular topology as described above, to acquire multi-channel sound signals from the physical environment in which the portable device is located. The specific installation positions, spacing between the three omnidirectional microphones, and spacing in the axial and lateral directions can be flexibly set by those skilled in the art according to the specific shape and size constraints of the portable device.

[0065] The audio processing chip 202 is communicatively connected to the triangular microphone array 204 and is used to process the sound signals acquired by the triangular microphone array 204 in real time. For example, it can read the corresponding executable instructions from the memory 206 to process the sound according to the sound pickup method described in the foregoing method embodiments, including differential processing of the sound signals acquired by the triangular microphone array 204 in the axial and lateral directions to obtain differential signals, and adaptive filtering based on the differential signals to obtain the target sound signal. The specific implementation of this processing can be referred to the relevant description in the foregoing method embodiments, and will not be repeated here. The audio processing chip 202 can be a dedicated audio digital signal processor, a general-purpose processor, a system-on-a-chip with integrated audio processing functions, an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA), etc. The specific implementation of the audio processing chip 202 is not limited in this application embodiment.

[0066] The memory 206 is communicatively connected to the audio processing chip 202 and stores at least one executable instruction. This executable instruction enables the audio processing chip 202 to perform the operations corresponding to the aforementioned sound pickup method based on the sound signals collected by the triangular microphone array 204. The memory 206 may include embedded non-volatile memory or similar materials for persistently storing the executable instruction and algorithm parameters. This application embodiment does not limit the specific type or storage capacity of the memory 206.

[0067] The specific implementation of the audio processing chip 202 in processing the sound signals collected by the triangular microphone array 204 can be referred to the relevant description in the foregoing method embodiments, and will not be repeated here.

[0068] Through this embodiment, the portable device can integrate the signal processing capabilities of the aforementioned sound pickup method into the portable device under relatively compact hardware constraints, achieving effective directional sound pickup and solving the problem that portable devices have difficulty clearly picking up the voice of the target speaker in complex acoustic environments.

[0069] This application also provides a computer storage medium storing a computer program that, when executed by a processor, implements the method described in any of the foregoing method embodiments. This computer storage medium includes, but is not limited to, embedded non-volatile memory and embedded static random access memory.

[0070] This application also provides a computer program product, including computer instructions that instruct a computing device to perform an operation corresponding to any of the sound pickup methods in the above-described multiple method embodiments.

[0071] Furthermore, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to user voice data, data for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0072] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0073] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0074] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A sound pickup method, comprising: Sound signals from the environment are collected using a triangular microphone array, which includes a first pair of microphones distributed along an axial direction and a second pair of microphones distributed along a transverse direction, wherein the axial direction and the transverse direction are perpendicular to each other. The sound signals collected by the microphone in the axial direction and the sound signals collected by the microphone in the lateral direction are subjected to differential processing to obtain differential signals; The target sound signal is obtained by adaptive filtering based on the differential signal.

2. The method according to claim 1, wherein, The three microphones are omnidirectional microphones; the first distance between the two microphones in the axial direction is not equal to the second distance between the two microphones in the lateral direction.

3. The method according to claim 2, wherein, The adaptive filtering based on the differential signal to obtain the target sound signal includes: The frequency band information of the sound signal is obtained, the frequency band includes a first frequency band, a second frequency band and a third frequency band, wherein the first frequency band is lower than the second frequency band, and the second frequency band is lower than the third frequency band; If the frequency band information indicates that the frequency band of the sound signal is the second frequency band, then the differential signal in the axial direction and the differential signal in the lateral direction are differentially processed, and adaptive filtering is performed based on the differential processing result to obtain the target sound signal.

4. The method according to claim 3, wherein, The adaptive filtering based on the differential signal to obtain the target sound signal further includes: If the frequency band information indicates that the frequency band where the sound signal is located is the first frequency band, then adaptive filtering is performed based on the differential signal corresponding to the larger of the first distance and the second distance to obtain the target sound signal; And / or, If the frequency band information indicates that the sound signal is located in the third frequency band, then the differential signal in the axial direction and the differential signal in the lateral direction are summed and averaged. Based on the sound signal obtained after averaging, adaptive filtering is performed to obtain the target sound signal.

5. The method according to claim 4, wherein, The process of summing and averaging the differential signals in the axial direction and the lateral direction, and then performing adaptive filtering on the resulting audio signal, includes: When the frequency band transitions from the second frequency band to the third frequency band, the differential signal in the axial direction and the differential signal in the lateral direction are summed and then averaged. The averaged audio signal is phase-aligned with the differential signal of the second frequency band. Adaptive filtering is performed on the phase-aligned audio signal.

6. The method according to any one of claims 1-5, wherein, The adaptive filtering based on the differential signal to obtain the target sound signal includes: Obtain the differential signal in the axial direction and the differential signal in the transverse direction; Calculate the spatial energy ratio of the differential signal in the axial direction and the differential signal in the transverse direction; Based on the spatial energy ratio, the tracking parameters of the adaptive filtering algorithm are determined; Based on the adaptive filtering algorithm and its corresponding tracking parameters, the differential signal in the axial direction is adaptively filtered to obtain the target sound signal.

7. The method according to claim 6, wherein, The adaptive filtering algorithm is a recursive least squares filtering algorithm; The determination of the tracking parameters of the adaptive filtering algorithm based on the spatial energy ratio includes: If the spatial energy ratio indicates that the differential signal in the lateral direction is dominant, then the forgetting factor in the least squares filtering algorithm is reduced. If the spatial energy ratio indicates that the differential signal in the axial direction is dominant, then the forgetting factor in the least squares filtering algorithm is increased.

8. The method according to claim 6, wherein, The adaptive filtering algorithm is the Kalman filtering algorithm; The determination of the tracking parameters of the adaptive filtering algorithm based on the spatial energy ratio includes: If the spatial energy ratio indicates that the differential signal in the lateral direction is dominant, then the process noise covariance in the Kalman filter algorithm is increased. If the spatial energy ratio indicates that the differential signal in the axial direction is dominant, then the process noise covariance in the Kalman filter algorithm is reduced.

9. A portable device comprising at least: Audio processing chip, triangular microphone array, and memory; The audio processing chip is communicatively connected to the triangular microphone array and the memory. The triangular microphone array is used to collect sound signals in the environment. The triangular microphone array includes a first pair of microphones distributed along the axial direction and a second pair of microphones distributed along the transverse direction. The axial direction and the transverse direction are perpendicular to each other. The memory is used to store at least one executable instruction, which causes the audio processing chip to perform the operation corresponding to the sound pickup method as described in any one of claims 1-8 based on the sound signal collected by the triangular microphone array.

10. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of claims 1-8.

11. A computer program product comprising computer instructions that instruct a computing device to perform an operation corresponding to any one of the methods described in claims 1-8.

Images