Directional audio transmission method and system based on beamforming

Abstract

The present invention provides a directional audio transmission method and system based on beamforming, which are applied to the field of audio data processing. The present invention can effectively distinguish and quantify the delay differences caused by different paths by performing delay estimation on pre-received audio signals and identifying delay difference data. At the same time, after detecting the multipath effect, activating a preset multi-channel receiving array can effectively capture signals from different directions. Based on these received signals, beamforming technology can be used to generate a beam pointing to a target direction, thereby suppressing interference signals from non-target directions. In addition, the weights of array elements are dynamically adjusted according to the target direction angle, which can further optimize the beamforming effect and ensure the clarity and strength of the target signal. This dynamic adjustment can cope with signal attenuation and interference changes caused by the multipath effect.

Description

Technical Field

[0001] The present invention relates to the field of audio data processing, and in particular to a directional audio transmission method and system based on beamforming. Background Art

[0002] With the popularity of music and video playback devices, people's entertainment life has been enriched. Currently, the existing music and video playback devices are mainly based on iOS and Android systems, so smart headphones that can be used for iOS and Android systems have also emerged. However, sound will encounter a multipath effect during transmission. Due to reflection, refraction or scattering, sound waves will reach the receiving point through multiple different paths. This phenomenon will cause signal interference and attenuation, thereby affecting the quality of sound transmission. Summary of the invention

[0003] The present invention aims to solve the problem that due to reflection, refraction or scattering, sound waves reach the receiving point through multiple different paths, causing signal interference and attenuation, thereby affecting the sound transmission quality, and provides a directional audio transmission method and system based on beamforming.

[0004] The present invention adopts the following technical means to solve the technical problem:

[0005] The present invention provides a directional audio transmission method based on beamforming, comprising:

[0006] Based on the audio signal content pre-received by the smart headset, the delay of the audio signal content is estimated, and the delay difference data of different paths is identified;

[0007] Determining whether the delay difference data meets the preset multipath effect characteristics;

[0008] If yes, activate the preset multi-channel receiving array, collect receiving signals from different directions, calculate the weights of each preset array element according to the receiving signals and the preset target direction, synthesize the weighted receiving signals, generate a beam pointing to the target direction, and iterate the beam generation process until the weights converge;

[0009] Determining whether the smart headset detects at least two closely spaced signal sources;

[0010] If detected, the covariance matrix of the received signal is calculated, and the eigenvalue decomposition is performed on the covariance matrix to obtain an eigenvector matrix and an eigenvalue matrix. According to the size of the eigenvalues ​​of the eigenvalue matrix, the eigenvector matrix is ​​divided into a signal subspace and a noise subspace. Spectral data is constructed based on the signal subspace and the noise subspace, and the peak position is searched in the spectral data to obtain the target direction angle of the closely spaced signal source, and the weights of the array elements are dynamically adjusted according to the target direction angle.

[0011] Furthermore, the step of synthesizing the weighted received signals to generate a beam pointing in the target direction includes:

[0012] Identify the array structure corresponding to the array element, calculate the array manifold vector based on the target direction and the array structure, obtain the relative phase and relative amplitude of the received signal from the array manifold vector, and calculate the weight of the array element according to the target direction and a preset beamforming algorithm, wherein the array structure specifically includes linear, circular and planar, and the beamforming algorithm specifically includes delay-sum beamforming and adaptive beamforming;

[0013] Determine whether the target signal needs to be enhanced;

[0014] If so, the array manifold vector is phase-aligned with the target signal, the distortion-free response of the target signal is maintained by minimizing the preset noise and interference variance, and the received signal is weighted with the weight to obtain the weighted received signal.

[0015] Furthermore, after the step of cyclically iterating the beam generation process until the weights converge, the step further includes:

[0016] Based on the array antenna type and weight distribution corresponding to the array elements, identifying the required main lobe width of the beam, wherein the array antenna type specifically includes a planar array antenna, a spherical array antenna, and a ring array antenna;

[0017] Determining whether the required main lobe width can suppress interference signals in the target direction;

[0018] If not, the main lobe width is adjusted accordingly according to the weight distribution, and the side lobe level of the beam is obtained at the same time, and the side lobe level is adjusted to a minimum level by reducing the height of the side lobe.

[0019] Furthermore, the step of dividing the eigenvector matrix into a signal subspace and a noise subspace, and constructing spectrum data based on the signal subspace and the noise subspace, further includes:

[0020] Constructing a beam weight vector based on the eigenvector matrix and the eigenvalue matrix, and maximizing the reception of the target signal by the smart headset with the beam weight vector;

[0021] Determining whether the beam weight vector can minimize a preset interference signal;

[0022] If not, the environment change information and signal characteristics pre-collected by the smart headset are identified, and the beam weight vector is dynamically adjusted according to the environment change information and the signal characteristics to maintain the beam pointing of the smart headset and the reception of the target signal.

[0023] Furthermore, the step of determining whether the delay difference data meets the preset multipath effect characteristics further includes:

[0024] Extracting corresponding statistical features from the delay difference data, and calculating a similarity measure between the statistical features and the multipath effect features, wherein the statistical features specifically include delay distribution features, frequency domain features, and waveform features;

[0025] Determining whether the similarity measure is greater than a preset similarity threshold;

[0026] If so, the delay difference data is divided into different categories based on the multipath effect characteristics, and data is annotated for each characteristic, wherein the different categories specifically include direct paths, reflection paths, and refraction paths, and the data annotations specifically include environmental conditions, signal source locations, number of paths, and delay values.

[0027] Further, after the step of calculating the covariance matrix of the received signal, performing eigenvalue decomposition on the covariance matrix, and obtaining an eigenvector matrix and an eigenvalue matrix, the method further includes:

[0028] Based on the eigenvalue decomposition of the covariance matrix, identifying the spatial spectrum peak value corresponding to the closely spaced signal source, and obtaining the direction information of the closely spaced signal source according to the spatial spectrum peak value;

[0029] Determining whether the direction information matches the estimated angle;

[0030] If so, based on the reachable angle of the preset microphone array, at least two position information of the closely spaced signal sources are calculated through geometric relationships, and the positioning information of the microphone array is fused to generate the relative position of the closely spaced signal sources.

[0031] Furthermore, the step of performing delay estimation on the audio signal content based on the audio signal content pre-received by the smart headset and identifying delay difference data of different paths includes:

[0032] Applying a bandpass filter preset by the smart headset to remove noise data in the audio signal content, retaining a valid signal within a preset target frequency range, wherein the noise data specifically includes background noise and interference signals;

[0033] Determining whether the valid signals are within the same amplitude range;

[0034] If not, the audio signal content is amplitude normalized, the peak value of the audio signal content is adjusted to a preset maximum value, the overall energy of the signal is adjusted based on the root mean square value of the audio signal content, and the audio signal content is scaled by decibels proportionally, wherein the amplitude normalization processing specifically includes peak normalization, mean square value normalization and decibel normalization.

[0035] The present invention also provides a directional audio transmission system based on beamforming, comprising:

[0036] An identification module, configured to estimate the delay of the audio signal content based on the audio signal content pre-received by the smart headset, and identify the delay difference data of different paths;

[0037] A judging module, used to judge whether the delay difference data meets the preset multipath effect characteristics;

[0038] an execution module, configured to, if yes, activate a preset multi-channel receiving array, collect receiving signals from different directions, calculate weights of each preset array element according to the receiving signals and a preset target direction, synthesize the weighted receiving signals, generate a beam pointing to the target direction, and iterate the beam generation process in a loop until the weights converge;

[0039] A second determination module, configured to determine whether the smart headset detects at least two closely spaced signal sources;

[0040] The second execution module is used to calculate the covariance matrix of the received signal if detected, perform eigenvalue decomposition on the covariance matrix to obtain an eigenvector matrix and an eigenvalue matrix, divide the eigenvector matrix into a signal subspace and a noise subspace according to the eigenvalue size of the eigenvalue matrix, construct spectrum data based on the signal subspace and the noise subspace, search for peak positions in the spectrum data, obtain the target direction angle of the closely spaced signal source, and dynamically adjust the weights of the array elements according to the target direction angle.

[0041] Furthermore, the execution module includes:

[0042] an identification unit, configured to identify an array structure corresponding to the array element, calculate an array manifold vector based on the target direction and the array structure, obtain a relative phase and a relative amplitude of the received signal from the array manifold vector, and calculate a weight of the array element according to the target direction and a preset beamforming algorithm, wherein the array structure specifically includes linear, circular and planar, and the beamforming algorithm specifically includes delay-sum beamforming and adaptive beamforming;

[0043] A judgment unit, used to judge whether the target signal needs to be enhanced;

[0044] An execution unit is used to, if yes, phase-align the array manifold vector with the target signal, maintain an undistorted response of the target signal by minimizing the preset noise and interference variance, and weight the received signal with the weight to obtain the weighted received signal.

[0045] Furthermore, it also includes:

[0046] A second identification module is used to identify the required main lobe width of the beam based on the array antenna type and weight distribution corresponding to the array element, wherein the array antenna type specifically includes a planar array antenna, a spherical array antenna and a ring array antenna;

[0047] A third judgment module is used to judge whether the required main lobe width can suppress the interference signal in the target direction;

[0048] The third execution module is used to adjust the main lobe width according to the weight distribution if no, and obtain the side lobe level of the beam at the same time, and adjust the side lobe level to the minimum level by reducing the height of the side lobe.

[0049] The present invention provides a directional audio transmission method and system based on beamforming, which has the following beneficial effects:

[0050] The present invention can effectively distinguish and quantify the delay differences caused by different paths by performing delay estimation and delay difference data identification on the pre-received audio signal. At the same time, after detecting the multipath effect, activating the preset multi-channel receiving array can effectively capture signals from different directions. Based on these received signals, beamforming technology can be used to generate a beam pointing in the target direction, thereby suppressing interference signals in non-target directions. In addition, the weights of the array elements are dynamically adjusted according to the target direction angle, which can further optimize the beamforming effect and ensure the clarity and strength of the target signal. This dynamic adjustment can cope with signal attenuation and interference changes caused by the multipath effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1The figure is a cross-sectional view of the overall structure of an embodiment of a directional audio transmission method based on beamforming of the present invention;

[0052] Figure 2 It is a partial structural diagram of an embodiment of a directional audio transmission system based on beamforming of the present invention. DETAILED DESCRIPTION

[0053] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The implementation of the objectives, functional features and advantages of the present invention will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Reference Figure 1 , a directional audio transmission method based on beamforming in an embodiment of the present invention, comprising:

[0056] S1: Based on the audio signal content pre-received by the smart headset, the delay of the audio signal content is estimated, and the delay difference data of different paths is identified;

[0057] S2: Determine whether the delay difference data meets the preset multipath effect characteristics;

[0058] S3: If yes, activate the preset multi-channel receiving array, collect receiving signals from different directions, calculate the weights of each preset array element according to the receiving signals and the preset target direction, synthesize the weighted receiving signals, generate a beam pointing to the target direction, and iterate the beam generation process until the weights converge;

[0059] S4: Determine whether the smart headset detects at least two closely spaced signal sources;

[0060] S5: If detected, calculate the covariance matrix of the received signal, perform eigenvalue decomposition on the covariance matrix to obtain an eigenvector matrix and an eigenvalue matrix, divide the eigenvector matrix into a signal subspace and a noise subspace according to the eigenvalue size of the eigenvalue matrix, construct spectrum data based on the signal subspace and the noise subspace, search for the peak position in the spectrum data, obtain the target direction angle of the closely spaced signal source, and dynamically adjust the weights of the array elements according to the target direction angle.

[0061] In this embodiment, the system estimates the delay of the audio signal content pre-received by the smart headset, identifies the delay difference data under different paths, and then determines whether the delay difference data meets the pre-set multipath effect characteristics to execute the corresponding steps; for example, when the system determines that the delay difference data of different paths cannot meet the pre-set multipath effect characteristics, the system will consider that the preset multipath effect feature library is not perfect and cannot cover all possible environmental changes and signal characteristics. The system will collect a large amount of audio data under different environments and conditions to cover as many scenarios and variations as possible, and extract new multipath effects from the new data. The system detects and adds the features to the feature library, uses statistical analysis methods to identify and record new feature patterns and abnormal conditions, and regularly performs self-checks on the hardware and software of the smart headset to ensure the normal operation of the device, record faults and abnormal conditions during device operation, and prevent similar problems from happening again. It also provides an interface for users to manually adjust and calibrate the system, allowing users to manually intervene and set parameters under special circumstances. For example, when the system determines that the delay difference data of different paths meets the pre-set multipath effect characteristics, the system will consider that the smart headset is affected by the multipath effect, resulting in interference attenuation of the signal. The system will activate the pre-set multi-channel receiving array to collect The system receives signals from different directions, calculates the weights of each preset array according to these received signals and the preset target direction, synthesizes the weighted received signals, generates a beam pointing to the target direction, and iterates the beam generation process until the weights converge. The system can receive signals from multiple directions by activating the multi-channel receiving array, which increases the diversity and richness of the signals. By weighted synthesis of the received signals, a beam pointing to the target direction is generated, which can enhance the strength of the target signal, suppress interference signals from other directions, and improve the clarity and quality of the signal. At the same time, the beamforming technology can be used to focus the signal to the target direction, reducing the interference caused by multipath propagation. The interference and attenuation caused by multipath effect can be effectively suppressed, and the influence of multipath effect on signal transmission quality can be effectively suppressed. By iterating the beam generation process until the weight converges, the beam forming effect can be continuously optimized to ensure that the final generated beam is accurately pointed and the interference suppression effect is better. The weights of each array are dynamically adjusted according to the received signal and the target direction, which can adapt to environmental changes and signal characteristics, improve the robustness and adaptability of the system, and immediately take corresponding measures to suppress interference and optimize signals when the multipath effect characteristics are detected, so as to improve the real-time and response speed of the system. Then the system determines whether the smart headset detects the existence of at least two closely spaced signal sources to execute the corresponding steps.For example, when the system determines that the smart headset has not detected at least two closely spaced signal sources, the system will believe that there are multiple signal sources, but their strengths are significantly different, and the interference of weak signal sources on strong signal sources is small, and will not significantly affect signal processing. The system will perform strength detection on the received signal, sort the signal sources according to their strength, identify strong signal sources and weak signal sources, and set the signal strength threshold according to the specific application scenario. Weak signal sources below the threshold are regarded as background noise or interference, and for the detected weak signal sources, a simplified processing flow is adopted to reduce the consumption of processing resources and ensure the processing efficiency of the main signal sources. According to the real-time detection results, the signal processing strategy and parameters are dynamically adjusted to optimize resource allocation to ensure efficient and accurate signal processing. For example, when the system determines that the smart headset has detected at least two closely spaced signal sources, the system will believe that there are multiple signal sources with large interference. The system will calculate the covariance matrix of the received signal, perform eigenvalue decomposition on the covariance matrix, obtain the eigenvector matrix and the eigenvalue matrix, and decompose the eigenvalue matrix according to the size of the eigenvalue of the eigenvalue matrix. The eigenvector matrix divides the signal subspace and the noise subspace, obtains the target direction angle of closely spaced signal sources, and dynamically adjusts the weights of the array elements according to these target direction angles; the system separates the signal subspace from the noise subspace through eigenvalue decomposition, which can effectively distinguish the main signal source from the background noise and reduce the interference of noise on signal processing. After separating the signal subspace, the closely spaced signal sources can be more accurately identified and processed, and the recognition ability of the target signal can be enhanced. At the same time, by analyzing the eigenvector matrix, the target direction angle of closely spaced signal sources can be obtained, and the direction of the signal source can be more accurately determined, and the accuracy of directional positioning can be improved. The weights of the array elements are dynamically adjusted according to the target direction angle, so that the array can focus on the target direction more accurately, further improving the positioning accuracy. And through the analysis of the eigenvalue, the characteristics of the main signal source can be identified, so as to effectively distinguish and process multiple closely spaced signal sources, reduce signal interference and overlap, and dynamically adjust the weights of the array elements, which can optimize the beamforming process, make the beam more focused on the target direction, and suppress interference signals from other directions. ;

[0062] It should be noted that a specific example of collecting received signals from different directions, calculating the weights of each preset array element according to the received signals and the preset target direction, synthesizing the weighted received signals, generating a beam pointing to the target direction, and iterating the beam generation process in a loop until the weights converge is as follows:

[0063] Assume that the smart headset has an array of 4 microphones. The system hopes to capture the main sound source signal from the front, while the noise and interference signals from other directions need to be suppressed as much as possible. The weight calculation formula is: ,in, It is The weights of the microphones, is the signal received by the microphone, is the target direction, and each microphone receives signals from different directions as follows,

[0064] Microphone 1: ;

[0065] Microphone 2: ;

[0066] Microphone 3: ;

[0067] Microphone 4: ;

[0068] The signal received by each microphone is multiplied by the calculated weight, and all weighted signals are synthesized. The synthesized signal will have strong directionality and point in the target direction, thereby realizing beamforming. The system presets the target direction as the front. Based on this direction, the weight of each microphone is calculated.

[0069] ;

[0070] ;

[0071] ;

[0072] ;

[0073] Assume that the initial weights are: , , and , multiply the signal received by each microphone by its weight and synthesize the final signal. The formula is as follows:

[0074] ;

[0075] in, is the synthesized signal, is the weight, It is The signal received by the microphone;

[0076] After the initial weight calculation, the system will generate a beam pointing to the target direction. However, the initial weight may not be optimal, so it is necessary to continuously optimize the weight through an iterative process. Through iterative optimization, assuming that the preset target signal is , then calculate the error between the synthetic signal and the target signal , adjust the weight of each microphone according to the error ,in, is the learning rate;

[0077] By repeating the above steps, until the error converges, that is, the weights no longer change significantly, until the final weights converge to the following,

[0078] ;

[0079] ;

[0080] ;

[0081] ;

[0082] To sum up, through the above steps, after the system converges the weights, the synthetic signal of the smart headset will be more directional, pointing to the target direction ahead. The interference signal can be effectively suppressed, and a beam pointing to the target direction can be generated, significantly improving the quality and directionality of signal processing.

[0083] As for calculating the covariance matrix of the received signal, the covariance matrix is ​​subjected to eigenvalue decomposition to obtain an eigenvector matrix and an eigenvalue matrix. According to the eigenvalue size of the eigenvalue matrix, the eigenvector matrix is ​​divided into a signal subspace and a noise subspace. Spectral data is constructed based on the signal subspace and the noise subspace. The peak position is searched in the spectral data to obtain the target direction angle of the closely spaced signal source. The weight of the array element is dynamically adjusted according to the target direction angle. The specific examples are as follows:

[0084] Assume that the smart headset has an array of 4 microphones and receives signals from different directions. The steps to calculate the covariance matrix are as follows:

[0085] Assume the received signal is:

[0086] ;

[0087] Then the covariance matrix R is calculated as: ,in, represents the expected value, yes The conjugate transpose of ;

[0088] Assume the calculation result is:

[0089] ;

[0090] Then the covariance matrix is ​​decomposed into eigenvalues ​​to obtain:

[0091] , ;

[0092] in, is the eigenvector matrix, is a diagonal eigenvalue matrix, the elements on the diagonal are eigenvalues, and according to the size of the eigenvalue, the eigenvector matrix It is divided into signal subspace and noise subspace. Assume that the eigenvectors corresponding to the first M largest eigenvalues ​​constitute the signal subspace, and the remaining eigenvectors constitute the noise subspace. Then the signal subspace matrix is , the noise subspace matrix ;

[0093] Assume that the eigenvalue is , , , ;

[0094] According to the size of the eigenvalue, the eigenvectors corresponding to the first two largest eigenvalues ​​are selected as the signal subspace, and the rest are used as the noise subspace;

[0095] The signal subspace matrix is:

[0096] ;

[0097] The noise subspace matrix is:

[0098] ;

[0099] Then use the MUSIC algorithm to construct the spectrum data, assuming ;

[0100] in, is the direction angle An array of manifold vectors, in the spectral function The peak position is searched in the spectrum data. The direction angle corresponding to the peak position is the target direction angle of the closely spaced signal source. By searching the peak position in the spectrum data, it is found that exist At the maximum value, therefore, the target direction angle is 45°. Dynamically adjust the weights of array elements to correspond to adjusting the weights of array elements , through the formula to achieve, and , then suppose for ;

[0101] The new weight , , , ;

[0102] In summary, through the above steps, the system can effectively separate and process closely spaced signal sources, optimize the directionality and quality of received signals, and improve signal processing effects.

[0103] In this embodiment, the step S3 of synthesizing the weighted received signals to generate a beam pointing to the target direction includes:

[0104] S31: Identify the array structure corresponding to the array element, calculate the array manifold vector based on the target direction and the array structure, obtain the relative phase and relative amplitude of the received signal from the array manifold vector, and calculate the weight of the array element according to the target direction and a preset beamforming algorithm, wherein the array structure specifically includes linear, circular and planar, and the beamforming algorithm specifically includes delay-sum beamforming and adaptive beamforming;

[0105] S32: Determine whether the target signal needs to be enhanced;

[0106] S33: If yes, then phase align the array manifold vector with the target signal, maintain the distortion-free response of the target signal by minimizing the preset noise and interference variance, and weight the received signal with the weight to obtain the weighted received signal.

[0107] In this embodiment, the system identifies the array structure corresponding to the array element, the array structure specifically includes linear, circular and planar, calculates the array manifold vector based on the target direction and the array structure, obtains the relative phase and relative amplitude of the received signal from the array manifold vector, calculates the weight of the array element according to the target direction and a pre-set beamforming algorithm, the beamforming algorithm specifically includes delayed sum beamforming and adaptive beamforming, and then the system determines whether the target signal needs to be enhanced to execute the corresponding steps; for example, when the system determines that the target signal does not need to be enhanced, the system will consider that the quality of the currently received signal has met the expected requirements or standards, and the system will continue to use the current array element weights and beamforming configuration without any additional adjustments to ensure the stability and consistency of signal processing, while continuously monitoring the signal quality and detecting possible changes. If the signal quality changes, the system can re-evaluate whether the target signal needs to be enhanced at any time, and record the current signal status and configuration to avoid unnecessary calculations and processing, save system resources, and reduce energy consumption; for example, when the system determines that the target signal needs to be enhanced When enhancement is required, the system will think that the quality of the currently received signal cannot meet the expected requirements or standards. The system will phase-align the array manifold vector with the target signal, maintain the distortion-free response of the target signal by pre-setting the minimum noise and interference variance, and weight the received signal with the weight to obtain the weighted received signal; the system can significantly enhance the target signal through phase alignment and weighted processing, so that its quality meets the expected requirements or standards. This process helps to clearly capture the target sound in a noisy environment. By minimizing the noise and interference variance, the interference and noise from other directions can be effectively reduced, thereby improving the clarity and quality of the target signal. At the same time, while performing signal enhancement, the system can maintain the distortion-free response of the target signal to ensure that the signal is not distorted after processing, which is particularly important for high-fidelity audio transmission. In the weighted processing process, the original characteristics of the target signal are retained, so that the final output signal is as consistent as possible with the original signal, and the weights can be dynamically adjusted according to the real-time received signal quality to ensure that the best signal enhancement effect can be provided in different environments.

[0108] It should be noted that the array structure corresponding to the array element is identified, an array manifold vector is calculated based on the target direction and the array structure, a relative phase and a relative amplitude of the received signal are obtained from the array manifold vector, and a weight of the array element is calculated according to the target direction and a preset beamforming algorithm. The specific examples are as follows:

[0109] Assume that the smart headset has a linear array consisting of 4 antenna elements with a spacing of , the received target signal direction is , and assuming that the target signal comes from the direction Arrival array, array manifold vector It can be expressed as:

[0110]

[0111] in, is the wavelength of the signal;

[0112] From the array manifold vector, the relative phase and relative amplitude of each received signal can be extracted. This information is used for subsequent beamforming calculations. The relative phase is The relative amplitude is usually set to 1, indicating that the signal strength received by each component is consistent, and the relative phase ;

[0113] Using delay-and-sum beamforming: ;

[0114] Use adaptive beamforming as: ;

[0115] Assume the target signal direction =30∘, signal wavelength = 1 m, antenna element spacing =0.5 m, then the array manifold vector is calculated as:

[0116]

[0117]

[0118] ;

[0119] And using delay-sum beamforming is:

[0120] ;

[0121] Using adaptive beamforming, assuming the covariance matrix is the identity matrix, ,for:

[0122] ;

[0123] In summary, the system can identify the array structure corresponding to the array elements, and calculate the array manifold vector based on the target direction and array structure, thereby obtaining the relative phase and relative amplitude of the received signal, and calculating the weights of the array elements according to the preset beamforming algorithm.

[0124] In this embodiment, after the step S3 of iterating the beam generation process until the weight converges, the following step is further included:

[0125] S301: Identify a required main lobe width of the beam based on an array antenna type and a weight distribution corresponding to the array element, wherein the array antenna type specifically includes a planar array antenna, a spherical array antenna, and a ring array antenna;

[0126] S302: Determine whether the required main lobe width can suppress interference signals in the target direction;

[0127] S303: If not, adjusting the main lobe width accordingly according to the weight distribution, acquiring the side lobe level of the beam, and adjusting the side lobe level to a minimum level by reducing the side lobe height.

[0128] In this embodiment, the system identifies the required main lobe width of the beam based on the array antenna type and weight distribution corresponding to the array elements, the array antenna types specifically including planar array antennas, spherical array antennas and annular array antennas, and then the system determines whether the main lobe width can suppress the interference signal in the target direction to execute the corresponding steps; for example, when the system determines that the required main lobe width of the beam can suppress the interference signal in the target direction, the system will consider that it has successfully generated a beam pointing in the target direction and with an appropriate main lobe width by adjusting the weight distribution of the array antenna, so that the interference signal is effectively suppressed, and the system will use the current weight distribution to perform weighted processing on the received signal, synthesize the output signal, and ensure that the output signal quality meets the expected standard, while monitoring the signal quality and interference in real time. If the environment changes or the interference signal increases, the system will dynamically adjust the weight distribution to maintain optimal performance; for example, when the system determines that the beam When the required main lobe width cannot suppress the interference signal in the target direction, the system will consider that the interference signal cannot be effectively suppressed, and the system will adjust the main lobe width according to the weight distribution, and obtain the sidelobe level of the beam at the same time, and adjust the sidelobe level to the minimum level by reducing the sidelobe height; the system adjusts the weight distribution to make the main lobe width more suitable for the target signal direction, further suppress the influence of the interference signal, reduce the sidelobe level, and reduce the interference signal from other directions entering the system, thereby improving the quality of the received signal. At the same time, the optimized beamforming makes the target signal more concentrated, enhances the strength and clarity of the target signal, reduces the influence of the interference signal and noise by reducing the sidelobe level, improves the overall signal quality, and optimizes the beamforming parameters. The system can maintain stable signal reception performance in complex environments, that is, the system can dynamically adjust the weight distribution and sidelobe level according to the real-time environment to adapt to different interference and noise environments.

[0129] It should be noted that the specific examples are as follows:

[0130] Assume that the system detects a signal at 45° in the target direction, but there is a strong interference signal from the 50° direction, and the initial beam main lobe width is 30°. In this case, the interference signal cannot be effectively suppressed, and the main lobe width needs to be optimized by adjusting the weight distribution. First, the weight distribution is calculated, and the weight is further adjusted to reduce the main lobe width to 20°, which is more concentrated in the target signal direction. Then the sidelobe level is obtained and adjusted. Since the initial sidelobe level of the system is high, the interference signal has a greater impact. Therefore, it is necessary to reduce the sidelobe height and adjust the sidelobe level to the minimum to further suppress the interference signal from the non-target direction.

[0131] In this embodiment, the eigenvector matrix is ​​divided into a signal subspace and a noise subspace, and the step S5 of constructing spectrum data based on the signal subspace and the noise subspace further includes:

[0132] S51: constructing a beam weight vector based on the eigenvector matrix and the eigenvalue matrix, and maximizing the reception of the target signal by the smart headset with the beam weight vector;

[0133] S52: Determine whether the beam weight vector can minimize a preset interference signal;

[0134] S53: If not, identifying the environmental change information and signal characteristics pre-collected by the smart headset, dynamically adjusting the beam weight vector according to the environmental change information and the signal characteristics, and maintaining the beam pointing of the smart headset and the reception of the target signal.

[0135] In this embodiment, the system constructs a beam weight vector based on the eigenvector matrix and the eigenvalue matrix, and uses the beam weight vector to maximize the reception of the target signal by the smart headset, and then the system determines whether the beam weight vector can minimize the pre-set interference signal to execute the corresponding steps; for example, when the system determines that the beam weight vector can minimize the pre-set interference signal, the system will consider that it has successfully optimized the beam weight vector so that it can effectively minimize the impact of the interference signal while maximizing the reception of the target signal. The system will ensure that the output signal quality meets the expected standard, use the current beam weight vector to weight the received signal, synthesize the output signal, and monitor the signal quality and interference in real time. If the environment changes or the interference signal increases, the system can dynamically adjust the beam weight vector to maintain optimal performance; for example, when the system determines that the beam weight vector cannot minimize the pre-set interference signal, the system will consider that it cannot be minimized. To minimize the impact of interference signals, the system will identify the environmental change signals and signal characteristics collected in advance by the smart headset, and dynamically adjust the beam weight vector according to these environmental change information and signal characteristics to maintain the beam pointing of the smart headset and the reception of the target signal; the system can dynamically adjust the beam weight vector according to environmental changes and signal characteristics to improve the system's adaptability in different environments. When the interference signal changes, the system can respond quickly and adjust the beam weight vector to continue to minimize the impact of interference. At the same time, it dynamically adjusts the beam weight vector to ensure that the reception quality of the target signal is always maintained in the best state under different environments and interference conditions. Through real-time adjustment, it can reduce signal distortion and bit error rate, improve the accuracy and reliability of signal processing, and in the process of dynamically adjusting the beam weight vector, ensure that the audio signal heard by the user is always clear and interference-free, so that the system can provide a consistent and high-quality listening experience no matter what environment the user is in.

[0136] In this embodiment, the step S2 of determining whether the delay difference data meets the preset multipath effect characteristics further includes:

[0137] S21: extracting corresponding statistical features from the delay difference data, and calculating a similarity measure between the statistical features and the multipath effect features, wherein the statistical features specifically include delay distribution features, frequency domain features, and waveform features;

[0138] S22: Determine whether the similarity measure is greater than a preset similarity threshold;

[0139] S23: If so, the delay difference data is divided into different categories based on the multipath effect characteristics, and data is annotated for each characteristic, wherein the different categories specifically include direct paths, reflection paths, and refraction paths, and the data annotations specifically include environmental conditions, signal source locations, number of paths, and delay values.

[0140] In this embodiment, the system extracts corresponding statistical features from the delay difference data, and the statistical features specifically include delay distribution features, frequency domain features, and waveform features, calculates similarity metrics between these statistical features and multipath effect features, and then the system determines whether these similarity metrics are greater than a preset similarity threshold to execute corresponding steps; for example, when the system determines that the similarity metrics between the statistical features and the multipath effect features are not greater than the preset similarity threshold, the system will consider that the signal propagation path in the current environment may be mainly a direct path, the multipath effect is not obvious, or the interference is small, the system will continue to monitor environmental changes, accumulate more data for analysis to ensure the accuracy of detection, regularly re-check the similarity metrics between the statistical features and the multipath effect features to ensure that the system can respond in time when the environment changes, and at the same time expand the multipath effect feature library to include more diverse features and cover a wider range of environmental and signal conditions. According to the actual situation of the current environment and signal, the existing feature library is adjusted and optimized to ensure its applicability and accuracy, and the received signal is filtered to eliminate possible abnormal signals and interference to improve signal quality; for example, when the system determines that If the similarity measure between the statistical feature and the multipath effect feature is greater than a preset similarity threshold, the system will consider that the multipath effect is obvious and the interference is large. The system will divide the delay difference data into different categories based on the multipath effect feature, and the different categories specifically include direct path, reflection path and refraction path, and perform data annotation on each feature, and the data annotation specifically includes environmental conditions, signal source location, number of paths and delay value; the system divides the delay difference data into direct path, reflection path and refraction path, which can more accurately identify and distinguish different signal paths, improve the accuracy of signal processing, and through classification processing, it can more effectively isolate and reduce the interference caused by different paths, and reduce the error rate. At the same time, through data annotation, the system can better understand and adapt to the signal propagation characteristics under different environmental conditions, and improve the environmental adaptability of the system. Based on detailed environmental and path information, the system can dynamically adjust and optimize the signal processing algorithm, improve the performance in complex environments, and identify and process various paths in the multipath effect, which can effectively reduce signal attenuation and distortion, ensure signal transmission quality, and improve the clarity of the received signal and improve user experience by optimizing the processing of different paths.

[0141] In this embodiment, after calculating the covariance matrix of the received signal and performing eigenvalue decomposition on the covariance matrix to obtain the eigenvector matrix and the eigenvalue matrix in step S5, the method further includes:

[0142] S501: identifying a spatial spectrum peak corresponding to the closely spaced signal source based on eigenvalue decomposition of the covariance matrix, and acquiring direction information of the closely spaced signal source according to the spatial spectrum peak;

[0143] S502: Determine whether the direction information matches the estimated angle;

[0144] S503: If yes, then according to the reachable angle of the preset microphone array, at least two position information of the closely spaced signal sources are calculated through geometric relationship, and the positioning information of the microphone array is integrated to generate the relative position of the closely spaced signal sources.

[0145] In this embodiment, the system identifies the spatial spectrum peak corresponding to the closely spaced signal source based on the eigenvalue decomposition of the covariance matrix, obtains the direction information of the closely spaced signal source according to the spatial spectrum peak, and then the system determines whether the direction information of the closely spaced signal source matches the estimated angle to execute the corresponding steps; for example, when the system determines that the direction information of the closely spaced signal source cannot match the estimated angle, the system will believe that the interference signal or multipath effect causes the direction information error, the system will regularly perform environmental scans, monitor environmental changes, such as the introduction of mobile obstacles or new equipment, to update the environmental model, and use environmental perception technology to reconstruct the signal path, especially in complex multipath situations, to correct the signal propagation path information, and at the same time, dynamically adjust the weight distribution of the array antenna according to the real-time signal measurement data to minimize the impact of the interference signal on the direction estimation, and increase the number and distribution of the receiving channels, and use multi-channel redundant information to improve the suppression ability of the interference signal; for example, when the system determines that the direction information of the closely spaced signal source can When the estimated angle is matched, the system will consider the direction information to be accurate. The system will calculate at least two position information of closely spaced signal sources based on the preset reachable angle of the microphone array and the geometric relationship, and fuse the positioning information of the microphone array to generate the relative position of the closely spaced signal source. After the system confirms the direction information of the closely spaced signal source by matching the estimated angle, it uses the preset reachable angle and geometric relationship of the microphone array to calculate the relative information of at least two positions of the closely spaced signal source. This accurate position estimation is very important for locating and tracking mobile signal sources. At the same time, through the fusion of the microphone array positioning information, it can effectively reduce the errors that may exist in the positioning process. Taking into account the information from multiple angles can improve the accuracy and stability of positioning. After obtaining the relative position information of the closely spaced signal source, the system can optimize the signal processing strategy based on this information, such as selecting an appropriate beamforming algorithm or signal tracking algorithm, to further improve the signal processing effect and the overall performance of the system.

[0146] It should be noted that the specific examples are as follows:

[0147] Suppose there is a circular microphone array with a preset reachable angle of 120 degrees and the center point of the array as the origin. Now a sound source is detected, and after signal processing, its main direction angle is found to be 30 degrees. First, the main direction angle is calculated. Since the main direction angle of the sound source is 30 degrees, the position is calculated based on the geometric relationship. Suppose the radius of the microphone array is 1 meter, the distance from the sound source to each microphone can be calculated by the time difference of the sound wave arrival. For example, if it takes 0.5 seconds for the first microphone to receive the signal and 0.6 seconds for the second microphone to receive the same signal, the distance difference between the sound source and the two microphones can be inferred. Based on these distance differences, the position of the sound source can be calculated using the triangulation method. Finally, the positioning information is fused. If other microphones also receive the same signal, their position information can be fused to improve the accuracy of positioning.

[0148] In this embodiment, based on the audio signal content pre-received by the smart headset, the delay of the audio signal content is estimated, and the step S1 of identifying the delay difference data of different paths includes:

[0149] S11: applying a bandpass filter preset by the smart headset to remove noise data in the audio signal content, and retaining a valid signal within a preset target frequency range, wherein the noise data specifically includes background noise and interference signals;

[0150] S12: Determine whether the valid signals are within the same amplitude range;

[0151] S13: If not, then perform amplitude normalization processing on the audio signal content, adjust the peak value of the audio signal content to a preset maximum value, adjust the overall energy of the signal based on the root mean square value of the audio signal content, and scale the audio signal content by decibels proportionally, wherein the amplitude normalization processing specifically includes peak normalization, mean square value normalization and decibel normalization.

[0152] In this embodiment, the system uses a bandpass filter pre-set in the smart headset to remove noise data in the audio signal content, the noise data specifically includes background noise and interference signals, and retains valid signals within a preset target frequency range. The system then determines whether these valid signals are within the same amplitude range to execute corresponding steps; for example, when the system determines that the valid signals within the target frequency range are within the same amplitude range, the system will consider that the audio signal maintains a stable and consistent strength within the preset target frequency range. The system will confirm the stability of the valid signal and continue with more advanced signal processing and analysis after the user selects it, including further noise reduction, audio enhancement or speech recognition, and retain the signal to the system for recording. A stable signal amplitude range can ensure the accuracy and consistency of transmission, and ensuring the stability and consistency of the signal helps to improve the stability and performance of the system, thereby providing more reliable results and responses in real-time applications; for example, when the system determines that When the valid signal within the target frequency range is not in the same amplitude range, the system will consider that the audio signal cannot maintain a stable and consistent intensity, and the system will perform amplitude normalization on the audio signal content. The amplitude normalization processing specifically includes peak normalization, mean square value normalization and decibel normalization. The peak value of the audio signal content is adjusted to a preset maximum value, and the overall energy of the signal is adjusted based on the root mean square value of the audio signal content, and the audio signal content is scaled proportionally by decibels. The system performs amplitude normalization to ensure that the audio signal maintains a consistent amplitude level throughout the entire processing process, which helps to avoid changes in audio quality or system instability caused by amplitude fluctuations. At the same time, by limiting the peak value of the audio signal to a preset maximum value, it can effectively prevent the signal from cracking or distortion during transmission or processing. Mean square value normalization of the signal can adjust the energy distribution of the signal, thereby improving the signal-to-noise ratio between the signal and the background noise, making the target signal more prominent and clear.

[0153] Reference Figure 2 , is a directional audio transmission system based on beamforming in an embodiment of the present invention, comprising:

[0154] The identification module 10 is used to estimate the delay of the audio signal content based on the audio signal content pre-received by the smart headset, and identify the delay difference data of different paths;

[0155] A judging module 20, configured to judge whether the delay difference data meets a preset multipath effect characteristic;

[0156] The execution module 30 is configured to, if yes, activate a preset multi-channel receiving array, collect receiving signals from different directions, calculate the weights of each preset array element according to the receiving signals and the preset target direction, synthesize the weighted receiving signals, generate a beam pointing to the target direction, and iterate the beam generation process until the weights converge;

[0157] A second determination module 40 is used to determine whether the smart headset detects at least two closely spaced signal sources;

[0158] The second execution module 50 is used to calculate the covariance matrix of the received signal if detected, perform eigenvalue decomposition on the covariance matrix to obtain an eigenvector matrix and an eigenvalue matrix, divide the eigenvector matrix into a signal subspace and a noise subspace according to the eigenvalue size of the eigenvalue matrix, construct spectrum data based on the signal subspace and the noise subspace, search for peak positions in the spectrum data, obtain the target direction angle of the closely spaced signal source, and dynamically adjust the weights of the array elements according to the target direction angle.

[0159] In this embodiment, the recognition module 10 estimates the delay of the audio signal content based on the audio signal content pre-received by the smart headset, identifies the delay difference data under different paths, and then the judgment module 20 judges whether the delay difference data meets the pre-set multipath effect characteristics to execute the corresponding steps; for example, when the system determines that the delay difference data of different paths cannot meet the pre-set multipath effect characteristics, the system will consider that the preset multipath effect feature library is not perfect and cannot cover all possible environmental changes and signal characteristics. The system will collect a large amount of audio data under different environments and conditions to cover as many scenarios and variations as possible, and extract new multipath effect features from the new data. Path effect features are added to the feature library, and statistical analysis methods are used to identify and record new feature patterns and abnormal conditions. At the same time, the hardware and software of the smart headset are self-checked regularly to ensure the normal operation of the equipment, record faults and abnormal conditions during equipment operation, and prevent similar problems from happening again. The system also provides users with an interface for manual adjustment and calibration of the system, allowing users to manually intervene and set parameters under special circumstances. For example, when the system determines that the delay difference data of different paths can meet the pre-set multipath effect features, the execution module 30 will consider that the smart headset is affected by the multipath effect, resulting in interference attenuation of the signal, and the system will activate the pre-set multi-channel receiving array. , collect receiving signals from different directions, calculate the weights of each preset array according to these receiving signals and the preset target direction, synthesize the weighted receiving signals, generate a beam pointing to the target direction, and iterate the beam generation process until the weights converge; the system can receive signals from multiple directions by activating the multi-channel receiving array, which increases the diversity and richness of the signals. By weighted synthesis of the received signals, a beam pointing to the target direction is generated, which can enhance the strength of the target signal, suppress interference signals from other directions, and improve the clarity and quality of the signal. At the same time, the beamforming technology can be used to focus the signal to the target direction and reduce the interference caused by multipath propagation. interference and attenuation, effectively suppressing the impact of multipath effects on signal transmission quality, and continuously optimizing the beamforming effect by iterating the beam generation process until the weights converge, ensuring that the final generated beam points accurately and has better interference suppression effect, and dynamically adjusting the weights of each array according to the received signal and the target direction, so as to adapt to environmental changes and signal feature changes, improve the robustness and adaptability of the system, and immediately take corresponding measures to suppress interference and optimize signals when it is detected that the multipath effect features are met, so as to improve the real-time performance and response speed of the system; then the second judgment module 40 judges whether the smart headset detects the presence of at least two closely spaced signal sources, so as to execute the corresponding steps;For example, when the system determines that the smart headset has not detected at least two closely spaced signal sources, the system will believe that there are multiple signal sources, but their strengths differ significantly, and the interference of weak signal sources on strong signal sources is small, and will not significantly affect signal processing. The system will perform strength detection on the received signal, sort the signal sources according to their strength, identify strong signal sources and weak signal sources, and set a signal strength threshold according to the specific application scenario, and regard weak signal sources below the threshold as background noise or interference. For the detected weak signal sources, a simplified processing flow is adopted to reduce the consumption of processing resources and ensure the processing efficiency of the main signal sources. According to the real-time detection results, the signal processing strategy and parameters are dynamically adjusted to optimize resource allocation to ensure efficient and accurate signal processing. For example, when the system determines that the smart headset has detected at least two closely spaced signal sources, the second execution module 50 will believe that there are multiple signal sources with large interference. The system will calculate the covariance matrix of the received signal, perform eigenvalue decomposition on the covariance matrix, obtain the eigenvector matrix and the eigenvalue matrix, and calculate the eigenvalue size of the eigenvalue matrix according to the eigenvalue matrix. , the eigenvector matrix is ​​divided into signal subspace and noise subspace, the target direction angle of closely spaced signal sources is obtained, and the weights of array elements are dynamically adjusted according to these target direction angles; the system separates the signal subspace from the noise subspace through eigenvalue decomposition, which can effectively distinguish the main signal source from the background noise and reduce the interference of noise on signal processing. After separating the signal subspace, the closely spaced signal sources can be more accurately identified and processed, and the recognition ability of the target signal can be enhanced. At the same time, by analyzing the eigenvector matrix, the target direction angle of closely spaced signal sources can be obtained, and the direction of the signal source can be more accurately determined, and the accuracy of directional positioning can be improved. The weights of the array elements are dynamically adjusted according to the target direction angle, so that the array can focus on the target direction more accurately, further improving the positioning accuracy, and through the analysis of the eigenvalues, the characteristics of the main signal source can be identified, thereby effectively distinguishing and processing multiple closely spaced signal sources, reducing signal interference and overlap, and dynamically adjusting the weights of the array elements can optimize the beamforming process, so that the beam is more focused on the target direction and suppresses interference signals from other directions. ;

[0160] In this embodiment, the execution module includes:

[0161] an identification unit, configured to identify an array structure corresponding to the array element, calculate an array manifold vector based on the target direction and the array structure, obtain a relative phase and a relative amplitude of the received signal from the array manifold vector, and calculate a weight of the array element according to the target direction and a preset beamforming algorithm, wherein the array structure specifically includes linear, circular and planar, and the beamforming algorithm specifically includes delay-sum beamforming and adaptive beamforming;

[0162] A judgment unit, used to judge whether the target signal needs to be enhanced;

[0163] An execution unit is used to, if yes, phase-align the array manifold vector with the target signal, maintain an undistorted response of the target signal by minimizing the preset noise and interference variance, and weight the received signal with the weight to obtain the weighted received signal.

[0164] In this embodiment, the system identifies the array structure corresponding to the array element, the array structure specifically includes linear, circular and planar, calculates the array manifold vector based on the target direction and the array structure, obtains the relative phase and relative amplitude of the received signal from the array manifold vector, calculates the weight of the array element according to the target direction and a pre-set beamforming algorithm, the beamforming algorithm specifically includes delayed sum beamforming and adaptive beamforming, and then the system determines whether the target signal needs to be enhanced to execute the corresponding steps; for example, when the system determines that the target signal does not need to be enhanced, the system will consider that the quality of the currently received signal has met the expected requirements or standards, and the system will continue to use the current array element weights and beamforming configuration without any additional adjustments to ensure the stability and consistency of signal processing, while continuously monitoring the signal quality and detecting possible changes. If the signal quality changes, the system can re-evaluate whether the target signal needs to be enhanced at any time, and record the current signal status and configuration to avoid unnecessary calculations and processing, save system resources, and reduce energy consumption; for example, when the system determines that the target signal needs to be enhanced When enhancement is required, the system will think that the quality of the currently received signal cannot meet the expected requirements or standards. The system will phase-align the array manifold vector with the target signal, maintain the distortion-free response of the target signal by pre-setting the minimum noise and interference variance, and weight the received signal with the weight to obtain the weighted received signal; the system can significantly enhance the target signal through phase alignment and weighted processing, so that its quality meets the expected requirements or standards. This process helps to clearly capture the target sound in a noisy environment. By minimizing the noise and interference variance, the interference and noise from other directions can be effectively reduced, thereby improving the clarity and quality of the target signal. At the same time, while performing signal enhancement, the system can maintain the distortion-free response of the target signal to ensure that the signal is not distorted after processing, which is particularly important for high-fidelity audio transmission. In the weighted processing process, the original characteristics of the target signal are retained, so that the final output signal is as consistent as possible with the original signal, and the weights can be dynamically adjusted according to the real-time received signal quality to ensure that the best signal enhancement effect can be provided in different environments.

[0165] In this embodiment, it also includes:

[0166] A second identification module is used to identify the required main lobe width of the beam based on the array antenna type and weight distribution corresponding to the array element, wherein the array antenna type specifically includes a planar array antenna, a spherical array antenna and a ring array antenna;

[0167] A third judgment module is used to judge whether the required main lobe width can suppress the interference signal in the target direction;

[0168] The third execution module is used to adjust the main lobe width according to the weight distribution if no, and obtain the side lobe level of the beam at the same time, and adjust the side lobe level to the minimum level by reducing the height of the side lobe.

[0169] In this embodiment, the system identifies the required main lobe width of the beam based on the array antenna type and weight distribution corresponding to the array elements, the array antenna types specifically including planar array antennas, spherical array antennas and annular array antennas, and then the system determines whether the main lobe width can suppress the interference signal in the target direction to execute the corresponding steps; for example, when the system determines that the required main lobe width of the beam can suppress the interference signal in the target direction, the system will consider that it has successfully generated a beam pointing in the target direction and with an appropriate main lobe width by adjusting the weight distribution of the array antenna, so that the interference signal is effectively suppressed, and the system will use the current weight distribution to perform weighted processing on the received signal, synthesize the output signal, and ensure that the output signal quality meets the expected standard, while monitoring the signal quality and interference in real time. If the environment changes or the interference signal increases, the system will dynamically adjust the weight distribution to maintain optimal performance; for example, when the system determines that the beam When the required main lobe width cannot suppress the interference signal in the target direction, the system will consider that the interference signal cannot be effectively suppressed, and the system will adjust the main lobe width according to the weight distribution, and obtain the sidelobe level of the beam at the same time, and adjust the sidelobe level to the minimum level by reducing the sidelobe height; the system adjusts the weight distribution to make the main lobe width more suitable for the target signal direction, further suppress the influence of the interference signal, reduce the sidelobe level, and reduce the interference signal from other directions entering the system, thereby improving the quality of the received signal. At the same time, the optimized beamforming makes the target signal more concentrated, enhances the strength and clarity of the target signal, reduces the influence of the interference signal and noise by reducing the sidelobe level, improves the overall signal quality, and optimizes the beamforming parameters. The system can maintain stable signal reception performance in complex environments, that is, the system can dynamically adjust the weight distribution and sidelobe level according to the real-time environment to adapt to different interference and noise environments.

[0170] In this embodiment, the second execution module further includes:

[0171] A construction unit, configured to construct a beam weight vector based on the eigenvector matrix and the eigenvalue matrix, so as to maximize the reception of the target signal by the smart headset with the beam weight vector;

[0172] A second judgment unit, used to judge whether the beam weight vector can minimize a preset interference signal;

[0173] The second execution unit is used to identify the environmental change information and signal characteristics pre-collected by the smart headset, dynamically adjust the beam weight vector according to the environmental change information and the signal characteristics, and maintain the beam pointing of the smart headset and the reception of the target signal.

[0174] In this embodiment, the system constructs a beam weight vector based on the eigenvector matrix and the eigenvalue matrix, and uses the beam weight vector to maximize the reception of the target signal by the smart headset, and then the system determines whether the beam weight vector can minimize the pre-set interference signal to execute the corresponding steps; for example, when the system determines that the beam weight vector can minimize the pre-set interference signal, the system will consider that it has successfully optimized the beam weight vector so that it can effectively minimize the impact of the interference signal while maximizing the reception of the target signal. The system will ensure that the output signal quality meets the expected standard, use the current beam weight vector to weight the received signal, synthesize the output signal, and monitor the signal quality and interference in real time. If the environment changes or the interference signal increases, the system can dynamically adjust the beam weight vector to maintain optimal performance; for example, when the system determines that the beam weight vector cannot minimize the pre-set interference signal, the system will consider that it cannot be minimized. To minimize the impact of interference signals, the system will identify the environmental change signals and signal characteristics collected in advance by the smart headset, and dynamically adjust the beam weight vector according to these environmental change information and signal characteristics to maintain the beam pointing of the smart headset and the reception of the target signal; the system can dynamically adjust the beam weight vector according to environmental changes and signal characteristics to improve the system's adaptability in different environments. When the interference signal changes, the system can respond quickly and adjust the beam weight vector to continue to minimize the impact of interference. At the same time, it dynamically adjusts the beam weight vector to ensure that the reception quality of the target signal is always maintained in the best state under different environments and interference conditions. Through real-time adjustment, it can reduce signal distortion and bit error rate, improve the accuracy and reliability of signal processing, and in the process of dynamically adjusting the beam weight vector, ensure that the audio signal heard by the user is always clear and interference-free, so that the system can provide a consistent and high-quality listening experience no matter what environment the user is in.

[0175] In this embodiment, the judgment module further includes:

[0176] A calculation unit, used to extract corresponding statistical features from the delay difference data, and calculate a similarity measure between the statistical features and the multipath effect features, wherein the statistical features specifically include delay distribution features, frequency domain features, and waveform features;

[0177] A third judgment unit, used to judge whether the similarity measure is greater than a preset similarity threshold;

[0178] The third execution unit is used to divide the delay difference data into different categories based on the multipath effect characteristics, and perform data annotation on each feature, wherein the different categories specifically include direct path, reflection path and refraction path, and the data annotation specifically includes environmental conditions, signal source location, number of paths and delay value.

[0179] In this embodiment, the system extracts corresponding statistical features from the delay difference data, and the statistical features specifically include delay distribution features, frequency domain features, and waveform features, calculates similarity metrics between these statistical features and multipath effect features, and then the system determines whether these similarity metrics are greater than a preset similarity threshold to execute corresponding steps; for example, when the system determines that the similarity metrics between the statistical features and the multipath effect features are not greater than the preset similarity threshold, the system will consider that the signal propagation path in the current environment may be mainly a direct path, the multipath effect is not obvious, or the interference is small, the system will continue to monitor environmental changes, accumulate more data for analysis to ensure the accuracy of detection, regularly re-check the similarity metrics between the statistical features and the multipath effect features to ensure that the system can respond in time when the environment changes, and at the same time expand the multipath effect feature library to include more diverse features and cover a wider range of environmental and signal conditions. According to the actual situation of the current environment and signal, the existing feature library is adjusted and optimized to ensure its applicability and accuracy, and the received signal is filtered to eliminate possible abnormal signals and interference to improve signal quality; for example, when the system determines that If the similarity measure between the statistical feature and the multipath effect feature is greater than a preset similarity threshold, the system will consider that the multipath effect is obvious and the interference is large. The system will divide the delay difference data into different categories based on the multipath effect feature, and the different categories specifically include direct path, reflection path and refraction path, and perform data annotation on each feature, and the data annotation specifically includes environmental conditions, signal source location, number of paths and delay value; the system divides the delay difference data into direct path, reflection path and refraction path, which can more accurately identify and distinguish different signal paths, improve the accuracy of signal processing, and through classification processing, it can more effectively isolate and reduce the interference caused by different paths, and reduce the error rate. At the same time, through data annotation, the system can better understand and adapt to the signal propagation characteristics under different environmental conditions, and improve the environmental adaptability of the system. Based on detailed environmental and path information, the system can dynamically adjust and optimize the signal processing algorithm, improve the performance in complex environments, and identify and process various paths in the multipath effect, which can effectively reduce signal attenuation and distortion, ensure signal transmission quality, and improve the clarity of the received signal and improve user experience by optimizing the processing of different paths.

[0180] In this embodiment, it also includes:

[0181] An acquisition module, configured to identify a spatial spectrum peak value corresponding to the closely spaced signal source based on eigenvalue decomposition of the covariance matrix, and acquire direction information of the closely spaced signal source according to the spatial spectrum peak value;

[0182] A fourth determination module, used to determine whether the direction information matches the estimated angle;

[0183] The fourth execution module is used to calculate at least two position information of the closely spaced signal sources through geometric relationships according to the reachable angle of the preset microphone array, fuse the positioning information of the microphone array, and generate the relative position of the closely spaced signal sources.

[0184] In this embodiment, the system identifies the spatial spectrum peak corresponding to the closely spaced signal source based on the eigenvalue decomposition of the covariance matrix, obtains the direction information of the closely spaced signal source according to the spatial spectrum peak, and then the system determines whether the direction information of the closely spaced signal source matches the estimated angle to execute the corresponding steps; for example, when the system determines that the direction information of the closely spaced signal source cannot match the estimated angle, the system will believe that the interference signal or multipath effect causes the direction information error, the system will regularly perform environmental scans, monitor environmental changes, such as the introduction of mobile obstacles or new equipment, to update the environmental model, and use environmental perception technology to reconstruct the signal path, especially in complex multipath situations, to correct the signal propagation path information, and at the same time, dynamically adjust the weight distribution of the array antenna according to the real-time signal measurement data to minimize the impact of the interference signal on the direction estimation, and increase the number and distribution of the receiving channels, and use multi-channel redundant information to improve the suppression ability of the interference signal; for example, when the system determines that the direction information of the closely spaced signal source can When the estimated angle is matched, the system will consider the direction information to be accurate. The system will calculate at least two position information of closely spaced signal sources based on the preset reachable angle of the microphone array and the geometric relationship, and fuse the positioning information of the microphone array to generate the relative position of the closely spaced signal source. After the system confirms the direction information of the closely spaced signal source by matching the estimated angle, it uses the preset reachable angle and geometric relationship of the microphone array to calculate the relative information of at least two positions of the closely spaced signal source. This accurate position estimation is very important for locating and tracking mobile signal sources. At the same time, through the fusion of the microphone array positioning information, it can effectively reduce the errors that may exist in the positioning process. Taking into account the information from multiple angles can improve the accuracy and stability of positioning. After obtaining the relative position information of the closely spaced signal source, the system can optimize the signal processing strategy based on this information, such as selecting an appropriate beamforming algorithm or signal tracking algorithm, to further improve the signal processing effect and the overall performance of the system.

[0185] In this embodiment, the identification module further includes:

[0186] A removal unit, configured to remove noise data in the audio signal content by applying a bandpass filter preset by the smart headset, and retain a valid signal within a preset target frequency range, wherein the noise data specifically includes background noise and interference signals;

[0187] A fourth determination unit, used to determine whether the valid signals are within the same amplitude range;

[0188] The fourth execution unit is used to, if not, perform amplitude normalization processing on the audio signal content, adjust the peak value of the audio signal content to a preset maximum value, adjust the overall energy of the signal based on the root mean square value of the audio signal content, and scale the audio signal content by decibels proportionally, wherein the amplitude normalization processing specifically includes peak normalization, mean square value normalization and decibel normalization.

[0189] In this embodiment, the system uses a bandpass filter pre-set in the smart headset to remove noise data in the audio signal content, the noise data specifically includes background noise and interference signals, and retains valid signals within a preset target frequency range. The system then determines whether these valid signals are within the same amplitude range to execute corresponding steps; for example, when the system determines that the valid signals within the target frequency range are within the same amplitude range, the system will consider that the audio signal maintains a stable and consistent strength within the preset target frequency range. The system will confirm the stability of the valid signal and continue with more advanced signal processing and analysis after the user selects it, including further noise reduction, audio enhancement or speech recognition, and retain the signal to the system for recording. A stable signal amplitude range can ensure the accuracy and consistency of transmission, and ensuring the stability and consistency of the signal helps to improve the stability and performance of the system, thereby providing more reliable results and responses in real-time applications; for example, when the system determines that When the valid signal within the target frequency range is not in the same amplitude range, the system will consider that the audio signal cannot maintain a stable and consistent intensity, and the system will perform amplitude normalization on the audio signal content. The amplitude normalization processing specifically includes peak normalization, mean square value normalization and decibel normalization. The peak value of the audio signal content is adjusted to a preset maximum value, and the overall energy of the signal is adjusted based on the root mean square value of the audio signal content, and the audio signal content is scaled proportionally by decibels. The system performs amplitude normalization to ensure that the audio signal maintains a consistent amplitude level throughout the entire processing process, which helps to avoid changes in audio quality or system instability caused by amplitude fluctuations. At the same time, by limiting the peak value of the audio signal to a preset maximum value, it can effectively prevent the signal from cracking or distortion during transmission or processing. Mean square value normalization of the signal can adjust the energy distribution of the signal, thereby improving the signal-to-noise ratio between the signal and the background noise, making the target signal more prominent and clear.

[0190] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A directional audio transmission method based on beamforming, characterized in that: The following steps are involved: Based on the audio signal content pre-received by the smart headset, the delay of the audio signal content is estimated, and the delay difference data of different paths is identified; Determining whether the delay difference data meets the preset multipath effect characteristics; If yes, activate the preset multi-channel receiving array, collect receiving signals from different directions, calculate the weights of each preset array element according to the receiving signals and the preset target direction, synthesize the weighted receiving signals, generate a beam pointing to the target direction, and iterate the beam generation process until the weights converge; Determining whether the smart headset detects at least two closely spaced signal sources; If detected, the covariance matrix of the received signal is calculated, and the eigenvalue decomposition is performed on the covariance matrix to obtain an eigenvector matrix and an eigenvalue matrix, and the eigenvector matrix is ​​divided into a signal subspace and a noise subspace according to the eigenvalue size of the eigenvalue matrix, and spectrum data is constructed based on the signal subspace and the noise subspace, and the peak position is searched in the spectrum data to obtain the target direction angle of the closely spaced signal source, and the weights of the array elements are dynamically adjusted according to the target direction angle; The step of synthesizing the weighted received signals to generate a beam pointing in the target direction includes: Identify the array structure corresponding to the array element, calculate the array manifold vector based on the target direction and the array structure, obtain the relative phase and relative amplitude of the received signal from the array manifold vector, and calculate the weight of the array element according to the target direction and a preset beamforming algorithm, wherein the array structure specifically includes linear, circular and planar, and the beamforming algorithm specifically includes delay-sum beamforming and adaptive beamforming; Determine whether the target signal needs to be enhanced; If so, the array manifold vector is phase-aligned with the target signal, the distortion-free response of the target signal is maintained by minimizing the preset noise and interference variance, and the received signal is weighted with the weight to obtain the weighted received signal.

2. The directional audio transmission method based on beamforming according to claim 1, characterized in that: After the step of cyclically iterating the beam generation process until the weight converges, the process further includes: Based on the array antenna type and weight distribution corresponding to the array elements, identifying the required main lobe width of the beam, wherein the array antenna type specifically includes a planar array antenna, a spherical array antenna, and a ring array antenna; Determining whether the required main lobe width can suppress interference signals in the target direction; If not, the main lobe width is adjusted accordingly according to the weight distribution, and the side lobe level of the beam is obtained at the same time, and the side lobe level is adjusted to the minimum level by reducing the height of the side lobe.

3. The directional audio transmission method based on beamforming according to claim 1, characterized in that: The step of dividing the eigenvector matrix into a signal subspace and a noise subspace, and constructing spectrum data based on the signal subspace and the noise subspace, further includes: Constructing a beam weight vector based on the eigenvector matrix and the eigenvalue matrix, and maximizing the reception of the target signal by the smart headset with the beam weight vector; Determining whether the beam weight vector can minimize a preset interference signal; If not, the environment change information and signal characteristics pre-collected by the smart headset are identified, and the beam weight vector is dynamically adjusted according to the environment change information and the signal characteristics to maintain the beam pointing of the smart headset and the reception of the target signal.

4. The directional audio transmission method based on beamforming according to claim 1, characterized in that: The step of determining whether the delay difference data meets the preset multipath effect characteristics also includes: Extracting corresponding statistical features from the delay difference data, and calculating a similarity measure between the statistical features and the multipath effect features, wherein the statistical features specifically include delay distribution features, frequency domain features, and waveform features; Determining whether the similarity measure is greater than a preset similarity threshold; If so, the delay difference data is divided into different categories based on the multipath effect characteristics, and data is annotated for each characteristic, wherein the different categories specifically include direct paths, reflection paths, and refraction paths, and the data annotations specifically include environmental conditions, signal source locations, number of paths, and delay values.

5. The directional audio transmission method based on beamforming according to claim 1, characterized in that: The covariance matrix of the received signal is calculated, and eigenvalue decomposition is performed on the covariance matrix. After the steps of obtaining the eigenvector matrix and the eigenvalue matrix, the following steps are also included: Based on the eigenvalue decomposition of the covariance matrix, identifying the spatial spectrum peak value corresponding to the closely spaced signal source, and obtaining the direction information of the closely spaced signal source according to the spatial spectrum peak value; Determining whether the direction information matches the estimated angle; If so, based on the reachable angle of the preset microphone array, at least two position information of the closely spaced signal sources are calculated through geometric relationships, and the positioning information of the microphone array is fused to generate the relative position of the closely spaced signal sources.

6. The beamforming-based directional audio transmission method according to claim 1, characterized in that: The step of performing delay estimation on the audio signal content based on the audio signal content pre-received by the smart headset and identifying the delay difference data of different paths includes: Applying a bandpass filter preset by the smart headset to remove noise data in the audio signal content, retaining a valid signal within a preset target frequency range, wherein the noise data specifically includes background noise and interference signals; Determining whether the valid signals are within the same amplitude range; If not, the audio signal content is amplitude normalized, the peak value of the audio signal content is adjusted to a preset maximum value, the overall energy of the signal is adjusted based on the root mean square value of the audio signal content, and the audio signal content is scaled by decibels proportionally, wherein the amplitude normalization processing specifically includes peak normalization, mean square value normalization and decibel normalization.

7. A directional audio transmission system based on beamforming, characterized in that: include: An identification module, configured to estimate the delay of the audio signal content based on the audio signal content pre-received by the smart headset, and identify the delay difference data of different paths; A judging module, used to judge whether the delay difference data meets the preset multipath effect characteristics; an execution module, configured to, if yes, activate a preset multi-channel receiving array, collect receiving signals from different directions, calculate weights of each preset array element according to the receiving signals and a preset target direction, synthesize the weighted receiving signals, generate a beam pointing to the target direction, and iterate the beam generation process in a loop until the weights converge; A second determination module, configured to determine whether the smart headset detects at least two closely spaced signal sources; A second execution module is used to calculate the covariance matrix of the received signal if detected, perform eigenvalue decomposition on the covariance matrix to obtain an eigenvector matrix and an eigenvalue matrix, divide the eigenvector matrix into a signal subspace and a noise subspace according to the eigenvalue size of the eigenvalue matrix, construct spectrum data based on the signal subspace and the noise subspace, search for peak positions in the spectrum data, obtain the target direction angle of the closely spaced signal source, and dynamically adjust the weights of the array elements according to the target direction angle; Wherein, the execution module includes: an identification unit, configured to identify an array structure corresponding to the array element, calculate an array manifold vector based on the target direction and the array structure, obtain a relative phase and a relative amplitude of the received signal from the array manifold vector, and calculate a weight of the array element according to the target direction and a preset beamforming algorithm, wherein the array structure specifically includes linear, circular and planar, and the beamforming algorithm specifically includes delay-sum beamforming and adaptive beamforming; A judgment unit, used to judge whether the target signal needs to be enhanced; An execution unit is used to, if yes, phase-align the array manifold vector with the target signal, maintain an undistorted response of the target signal by minimizing the preset noise and interference variance, and weight the received signal with the weight to obtain the weighted received signal.

8. The beamforming-based directional audio transmission system according to claim 7, characterized in that: Also includes: A second identification module is used to identify the required main lobe width of the beam based on the array antenna type and weight distribution corresponding to the array element, wherein the array antenna type specifically includes a planar array antenna, a spherical array antenna and a ring array antenna; A third judgment module is used to judge whether the required main lobe width can suppress the interference signal in the target direction; The third execution module is used to adjust the main lobe width according to the weight distribution if no, and obtain the side lobe level of the beam at the same time, and adjust the side lobe level to the minimum level by reducing the height of the side lobe.

Images